# Given marginal of $U$ and $V$, how can we know if there is joint such that $\mathbb E[U|V]=V$

Suppose we are given two random variable $$U$$ and $$V$$ on the interval $$[0,1]$$ with respective probability measure $$p_U$$ and $$p_V$$.

When is there a coupling measure $$p_{U,V}$$, that respect the marginals $$p_U$$ and $$p_V$$, such that $$\mathbb E[U|V]=V$$.

I have tried few approach, like expressing this as a linear program and looking at the dual, but the following seemed to be the most promising.

Suppose we have a measurable convex set $$S\subseteq [0,1]$$, if we are given $$V$$, then what is the maximal amount of probability we can give to $$U\in S$$. Observe that the conditional expectation constraint gives \begin{align*} V &= \mathbb E[U|V]\\ &=\mathbb E[1_S(U)|V]\cdot \inf_{a\in S}a+\mathbb E[(U-\inf_{a\in S} a) 1_S(U)|V]+\mathbb E[U1_{S^c}(U)|V]\\ &\geq \mathbb E[1_S(U)|V]\cdot \inf_{a\in S}a \end{align*} So that we have $$\mathbb E[1_S(U)|V]\leq\sup_a\frac{V}{a}$$, similarly, we can also get \begin{align*} 1-V&= 1-\mathbb E[U|V]\\ &\geq \mathbb E[1_S(U)|V]\cdot \inf_{a\in S}(1-a) \end{align*} So that $$\mathbb E[1_S(U)|V]\leq\sup_a\frac{1-V}{1-a}$$, and finally we also know that $$\mathbb E[1_S(U)|V]\leq 1$$.

If we average we get that \begin{align*} P_U(S) &= \mathbb E[\mathbb E[1_S(U)|V]]\\ &\leq \mathbb E\left[ \min\left(\sup_{a\in S}\frac{V}{a},\sup_{a\in S}\frac{1-V}{1-a},1\right) \right] \end{align*} Now if we have $$\mathbb E[U|V]=V$$, this needs to be satisfied for all $$S$$, but it feels like if we add the constraint that $$\mathbb E[U]=\mathbb E[V]$$, then we get that this is an if and only if. In order to prove this I don't think I have another choice than giving the conditional $$p_{U|V}$$ but I am not sure how to do that. It would also be nice to have a modification of that constraint that will also enforce that $$\mathbb E[U]=\mathbb E[V]$$ (because I don't think it is the case right now).

In the following I will try to motivate the problem and explain why it would be useful to solve. In short it would improve a lot my (and hopefully other's) understanding of Markov chains.

Let us be given a random variable $$X$$ with support $$\mathcal X$$. For $$x\in\mathcal X$$, define $$\delta_x$$ as the probability measure that gives $$1$$ to any set containing $$x$$ and $$0$$ otherwise. We can see $$p_{X|X}=\delta_X$$ as a random measure. for any random variable $$U$$, let $$p_{X|U}=\mathbb E[p_{X|X}|U]$$ where this conditional expectation is defined as $$\forall A$$, $$p_{X|U}(A)=\mathbb E[p_{X|X}(A)|U]$$.

We can prove a lot of things of those object and we can convince ourselves that this is indeed a conditional distribution since conditioned on $$U=u$$, we get that $$p_{X|U}$$ is the measure $$p_{X|U=u}$$. We can also show that this is a minimally sufficient statistic i.e. $$X-U-p_{X|U}$$ and $$X-p_{X|U}-U$$ are both Markov chains plus for any other sufficient statistic with $$X-U-V$$ and $$X-V-U$$, we have that $$p_{X|U}=p_{X|V}$$ almost surely (hence minimality).

Now of course $$p_{X|U}$$ is a random variable on the simplex of $$\mathcal X$$ with some distribution $$\mu$$, similarly we can have $$p_{X|V}$$ with distribution $$\nu$$ and I think it is important to wonder about when there can be a Markov chain $$X-p_{X|U}-p_{X|V}$$ by looking only at the distribution $$\mu$$ and $$\nu$$. And this is answered by

Is there a joint measure $$\lambda$$ with marginals $$\mu$$ and $$\nu$$ such that $$\mathbb E[p_{X|U}|V]=p_{X|V}$$.

And hence my question (which corresponds to the $$X$$ binary case), I think if I can solve the binary cases, there is chances the solution generalizes.

This is the subject of chapter 15 of Phelps's 1966 Lectures on Choquet's Theorem. The answer is, this happens exactly when the distributions of $$U$$ and $$V$$ are such that $$E[f(U)]\ge E[f(V)]$$ for all continuous convex functions $$f$$. There is a whole series of results, by Hardy, Littlewood, Polya, Blackwell, Stein, Sherman, Cartier, etc; what you want is contained in Blackwell's 1953 "Equivalent comparisons of experiments", Ann. Math. Statistics 24 (1953).

The buzz word is that the distribution of $$U$$ is a dilation of that of $$V$$, written $$\mu_U\succ\mu_V$$; the intuition is that the distribution of $$U$$ is more "spread out" than that of $$V$$. It is, in a sense, a kind of converse to Jensen's inequality.

To check the condition it suffices to restrict yourself to $$f$$ of form $$f(x)=\max(0,ax+b).$$

As the discussion in the comments show, the result (the "Blackwell-Sherman-Stein theorem") is not easy to prove, and I don't know of a simple way to understand (or explain) it. You might find these references helpful: The result is Theorem 2 in V. Strassen's 1965 paper The existence of probability measures with given marginals. The 1996 survey paper by Le Cam might also be useful.

• I do not fully understand the proof, do you think you could rephrase it in notations that are more up to date (and if possible that matches mine) ? I think the forum would benefit from having such a statement proved somewhere. I wonder if there is a way to generalize this trace of $UD$ to an operator that also works for infinite alphabets so that we don't have to use doob's martingale theorem. What do you think ? Nov 7, 2020 at 11:47
• PS : The proof confusing me is the one of Theorem 4, I don't really see how it relates to the convex functions conditions. Nov 7, 2020 at 11:58
• Phelp's book, and the paper by Cartier, etc (numdam.org/article/BSMF_1964__92__435_0.pdf) are more modern, and, with application, followable. I am an occasional user of this theory, but don't feel I am qualified to write a survey pitched to your precise level. Blackwell and Cartier suggest one starts at page 45 of Hardy, Littlewood, and Polya for the simplest case of the result. Nov 7, 2020 at 12:47
• Yes I just took a look at Phelb's, it is really good, a bit advanced for me but I can follow. Thank you very much for the other references, if I get anything clean I will write another answer but for now I will give you the bounty. Nov 7, 2020 at 16:37
• I wonder if there is a way that the following is true : for any random variable $U$ with measure $\mu$, there exists a convex function $f_\mu$ such that $\mathbb E[f_\mu(U)]\geq \mathbb E[f_\mu(V)]$ is equivalent to $V$ is a dilatation of $U$. For sure such a function would have to take value $+\infty$ on any point not in the support of $U$, intuitively it also feels like such a function cannot be linear at points on the support of $\mu$. Have you seen anything like that ? Nov 8, 2020 at 12:03