What is a good category for probability theory? I am searching for a good category to think about probability theory in, with arrows as something like stochastic maps. There are certain nice structural features I would like the category to have (listed below) and I was hoping somebody could tell me if there are any categories meeting a lot of these conditions:

*

*A monoidal category with a tensor product corresponding to something like the Cartesian product. Actually it would be good if it was a Markov category (like $\text{FinStoch}$ is for example).


*In addition to having objects corresponding to finite sets/spaces it would be good to have infinite sets/spaces appearing as objects. Ideally I want to be able to talk about continuous probability distributions (using real numbers etc.) in addition to discrete probability distributions.


*It would be great if I can form disjoint unions or coproducts of objects. Ideally I would like the category to be a bimonoidal category / rig category with $ \otimes $ corresponding to the Cartesian product and $\oplus$ corresponding to the disjoint union.


*It would be great if the deterministic maps/functions from something like Set are included in the category too, so I can think about functional programming and recursion there too.
I realize I am asking for a lot, but there are many different categories for doing probability theory which are mentioned in the literature, so I was hoping somebody could highlight some categories which get close to fulfilling my conditions. In particular there seem to be many categories such as $\text{FinStoch}$, $\text{Stoch}$, $\text{CGStoch}$ and $\text{BorelStoch}$ defined by Fong and Fritz. Also 28:38 of Perrone's YouTube video defines more categories (like the category of finitary stochastic maps, which is the Kleisli category of Set using the distribution monad, and may meet some of my conditions).
 A: This is an interesting question as there are a few perspectives we can take. In this answer, I'll provide some insight in the stochastic and causal cases. Whilst I am unaware of a category that meets all four criteria, there do exist categories that meet at least two of them.
From a stochastic perspective, the category $\sf FinStoch$ is a good starting point because it is the category of Markov kernels between finite sets so Criterion 1. is met. However it is only mostly used in the discrete case; for instance, in information theory/Bayesian networks, it can be applied in the formulation of the discrete (Shannon) entropy and divergence criteria. Other than the PhD thesis by Fong (2012), the Master's thesis by Rischel (2020) is also quite helpful.
More recently, Shiebler (2021) discovered a connection between categorical models and maximum likelihood estimation, developing from Fong (2012). This used the co-Kleisli category $\sf CoKl$ under the co-monad $(A\otimes\_)$ where $A$ is an object in the Cartesian monoidal category $C$. In the special case of the real space $\Bbb R^n$, this is termed $\sf CEuc$. This answers to Criterion 1. and Criterion 2. as the setting is wholly continuous.
As a bonus to Criterion 1., the related category $\sf PEuc$ (allowing stochastic processes in more than one probability space) was shown to be extendable to a Markov category. This is shown in Proposition 5, where each object is equipped with the comultiplication map $cp$ and the counit map $dc$ such that \begin{align}cp:1\times\Bbb R^a\to\Bbb R^a\times\Bbb R^a\quad&\text{where}\quad cp(-,x_a)=(x_a,x_a)\\dc:1\times\Bbb R^a\to1\quad&\text{where}\quad dc(-,x_a)=-.\end{align}
To give a reasonably close answer to Criterion 3., we can link the areas of stochastic theory and metric spaces together. The Kantorovich monad comes into mind, and the details are provided in the PhD thesis by Perrone (2018). By equipping the category (derived from $\sf 
CMet$, the category of complete metric spaces) with both lax morphisms (convex maps) and oplax morphisms (concave maps), it is possible to construct a hexagon equation to show bimonoidality. This is given in Proposition 2.5.16. Note Criterion 2. is also satisfied.
I don't know of a useful category that obeys Criterion 4. and at least one other criterion. In terms of recursion though, it could be worthwhile to study the backpropagation functor through $\sf Learn$ as each set of morphisms in the learning algorithm may be transferrable to models involving Bayesian updating, given a well-defined initial state.
With this literature being discussed, I think the Markov extension of $\sf PEuc$ would be of most relevance to the question and hopefully Shiebler's work will be further developed in the near future.
