2
$\begingroup$

Suppose we are given two random variable $U$ and $V$ on the interval $[0,1]$ with respective probability measure $p_U$ and $p_V$.

When is there a coupling measure $p_{U,V}$, that respect the marginals $p_U$ and $p_V$, such that $\mathbb E[U|V]=V$.

I have tried few approach, like expressing this as a linear program and looking at the dual, but the following seemed to be the most promising.

Suppose we have a measurable convex set $S\subseteq [0,1]$, if we are given $V$, then what is the maximal amount of probability we can give to $U\in S$. Observe that the conditional expectation constraint gives \begin{align*} V &= \mathbb E[U|V]\\ &=\mathbb E[1_S(U)|V]\cdot \inf_{a\in S}a+\mathbb E[(U-\inf_{a\in S} a) 1_S(U)|V]+\mathbb E[U1_{S^c}(U)|V]\\ &\geq \mathbb E[1_S(U)|V]\cdot \inf_{a\in S}a \end{align*} So that we have $\mathbb E[1_S(U)|V]\leq\sup_a\frac{V}{a}$, similarly, we can also get \begin{align*} 1-V&= 1-\mathbb E[U|V]\\ &\geq \mathbb E[1_S(U)|V]\cdot \inf_{a\in S}(1-a) \end{align*} So that $\mathbb E[1_S(U)|V]\leq\sup_a\frac{1-V}{1-a}$, and finally we also know that $\mathbb E[1_S(U)|V]\leq 1$.

If we average we get that \begin{align*} P_U(S) &= \mathbb E[\mathbb E[1_S(U)|V]]\\ &\leq \mathbb E\left[ \min\left(\sup_{a\in S}\frac{V}{a},\sup_{a\in S}\frac{1-V}{1-a},1\right) \right] \end{align*} Now if we have $\mathbb E[U|V]=V$, this needs to be satisfied for all $S$, but it feels like if we add the constraint that $\mathbb E[U]=\mathbb E[V]$, then we get that this is an if and only if. In order to prove this I don't think I have another choice than giving the conditional $p_{U|V}$ but I am not sure how to do that. It would also be nice to have a modification of that constraint that will also enforce that $\mathbb E[U]=\mathbb E[V]$ (because I don't think it is the case right now).


In the following I will try to motivate the problem and explain why it would be useful to solve. In short it would improve a lot my (and hopefully other's) understanding of Markov chains.

Let us be given a random variable $X$ with support $\mathcal X$. For $x\in\mathcal X$, define $\delta_x$ as the probability measure that gives $1$ to any set containing $x$ and $0$ otherwise. We can see $p_{X|X}=\delta_X$ as a random measure. for any random variable $U$, let $p_{X|U}=\mathbb E[p_{X|X}|U]$ where this conditional expectation is defined as $\forall A$, $p_{X|U}(A)=\mathbb E[p_{X|X}(A)|U]$.

We can prove a lot of things of those object and we can convince ourselves that this is indeed a conditional distribution since conditioned on $U=u$, we get that $p_{X|U}$ is the measure $p_{X|U=u}$. We can also show that this is a minimally sufficient statistic i.e. $X-U-p_{X|U}$ and $X-p_{X|U}-U$ are both Markov chains plus for any other sufficient statistic with $X-U-V$ and $X-V-U$, we have that $p_{X|U}=p_{X|V}$ almost surely (hence minimality).

Now of course $p_{X|U}$ is a random variable on the simplex of $\mathcal X$ with some distribution $\mu$, similarly we can have $p_{X|V}$ with distribution $\nu$ and I think it is important to wonder about when there can be a Markov chain $X-p_{X|U}-p_{X|V}$ by looking only at the distribution $\mu$ and $\nu$. And this is answered by

Is there a joint measure $\lambda$ with marginals $\mu$ and $\nu$ such that $\mathbb E[p_{X|U}|V]=p_{X|V}$.

And hence my question (which corresponds to the $X$ binary case), I think if I can solve the binary cases, there is chances the solution generalizes.

$\endgroup$

1 Answer 1

2
+250
$\begingroup$

This is the subject of chapter 15 of Phelps's 1966 Lectures on Choquet's Theorem. The answer is, this happens exactly when the distributions of $U$ and $V$ are such that $E[f(U)]\ge E[f(V)]$ for all continuous convex functions $f$. There is a whole series of results, by Hardy, Littlewood, Polya, Blackwell, Stein, Sherman, Cartier, etc; what you want is contained in Blackwell's 1953 "Equivalent comparisons of experiments", Ann. Math. Statistics 24 (1953).

The buzz word is that the distribution of $U$ is a dilation of that of $V$, written $\mu_U\succ\mu_V$; the intuition is that the distribution of $U$ is more "spread out" than that of $V$. It is, in a sense, a kind of converse to Jensen's inequality.

To check the condition it suffices to restrict yourself to $f$ of form $f(x)=\max(0,ax+b).$

As the discussion in the comments show, the result (the "Blackwell-Sherman-Stein theorem") is not easy to prove, and I don't know of a simple way to understand (or explain) it. You might find these references helpful: The result is Theorem 2 in V. Strassen's 1965 paper The existence of probability measures with given marginals. The 1996 survey paper by Le Cam might also be useful.

$\endgroup$
6
  • $\begingroup$ I do not fully understand the proof, do you think you could rephrase it in notations that are more up to date (and if possible that matches mine) ? I think the forum would benefit from having such a statement proved somewhere. I wonder if there is a way to generalize this trace of $UD$ to an operator that also works for infinite alphabets so that we don't have to use doob's martingale theorem. What do you think ? $\endgroup$
    – P. Quinton
    Nov 7, 2020 at 11:47
  • $\begingroup$ PS : The proof confusing me is the one of Theorem 4, I don't really see how it relates to the convex functions conditions. $\endgroup$
    – P. Quinton
    Nov 7, 2020 at 11:58
  • $\begingroup$ Phelp's book, and the paper by Cartier, etc (numdam.org/article/BSMF_1964__92__435_0.pdf) are more modern, and, with application, followable. I am an occasional user of this theory, but don't feel I am qualified to write a survey pitched to your precise level. Blackwell and Cartier suggest one starts at page 45 of Hardy, Littlewood, and Polya for the simplest case of the result. $\endgroup$ Nov 7, 2020 at 12:47
  • $\begingroup$ Yes I just took a look at Phelb's, it is really good, a bit advanced for me but I can follow. Thank you very much for the other references, if I get anything clean I will write another answer but for now I will give you the bounty. $\endgroup$
    – P. Quinton
    Nov 7, 2020 at 16:37
  • $\begingroup$ I wonder if there is a way that the following is true : for any random variable $U$ with measure $\mu$, there exists a convex function $f_\mu$ such that $\mathbb E[f_\mu(U)]\geq \mathbb E[f_\mu(V)]$ is equivalent to $V$ is a dilatation of $U$. For sure such a function would have to take value $+\infty$ on any point not in the support of $U$, intuitively it also feels like such a function cannot be linear at points on the support of $\mu$. Have you seen anything like that ? $\endgroup$
    – P. Quinton
    Nov 8, 2020 at 12:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .