6
$\begingroup$

Can conditional distributions determine the joint distribution?

For example, let $X_1, \dots, X_n$ be random variables.

Can their joint distribution be determined from the conditional distribution of $X_i$ given others, $i=1, \dots, n$?

Can their joint distribution be determined from other types of conditional distributions, such as the type of $P(X_i|X_j)$, and/or the type of $P(X_i, X_j | \text{others})$, and/or other types?

Thanks!

$\endgroup$
  • $\begingroup$ Except in the important case of independence, the answer to the first question is no. One can get sometimes useful inequalities. If more information is given, one can get the joint distribution. As a simple example, suppose that we are dealing with discrete distibutions, and we know $\Pr(X=a)$ and $\Pr(Y=b|X=a)$ for every $a$ and $b$. Then by the definition of conditional probability, we know $\Pr((X= a)\cap (Y=b))$, that is, we know the joint distribution. Similar considerations hold for conditional densities. $\endgroup$ – André Nicolas Mar 4 '14 at 16:26
  • $\begingroup$ @AndréNicolas: Thanks! Here is how my questions arose. In Gibbs sampling, the conditional distributions of Xi given others, i=1,…,n are called full conditional distributions. I heard that the joint distribution can be determined by all the full conditionals, but I am not sure if that is correct. $\endgroup$ – Tim Mar 4 '14 at 21:34
  • 5
    $\begingroup$ The answer to the first question is yes, without independence hypothesis. When densities exist, for two random variables $X$ and $Y$, note that $$f_{X\mid Y}(x\mid y)f_Y(y)=f_{X,Y}(x,y)=f_{Y\mid X}(y\mid x)f_X(x)$$ hence, $$f_Y(y)\int\frac{f_{X\mid Y}(x\mid y)}{f_{Y\mid X}(y\mid x)}dx=\int f_X(x)dx=1$$ hence $$f_Y(y)=\left(\int\frac{f_{X\mid Y}(x\mid y)}{f_{Y\mid X}(y\mid x)}dx\right)^{-1}$$ and $f_{X,Y}$ is fully determined by $f_{X\mid Y}$ and $f_{X\mid Y}$ through the formula $$f_{X,Y}(x,y)=f_{X\mid Y}(x\mid y)\left(\int\frac{f_{X\mid Y}(x\mid z)}{f_{Y\mid X}(z\mid x)}dz\right)^{-1}.$$ $\endgroup$ – Did Apr 30 '16 at 13:45
  • 1
    $\begingroup$ @Did is the variable replacement in last line correct? $\endgroup$ – Alexander Gruber Feb 1 '18 at 17:56
  • 1
    $\begingroup$ @AlexanderGruber It is not, in the integral one should replace the content of the parenthesis by $$\int\frac{f_{X\mid Y}(z\mid y)}{f_{Y\mid X}(y\mid z)}dz$$ Well spotted, thanks. $\endgroup$ – Did Feb 1 '18 at 19:36
3
$\begingroup$

This can in fact be true, and is very important and relevant in the context of Gibbs sampling as mentioned. If a joint distribution $P_{XY}$ has two conditional distributions $P_{X \mid Y}$ and $P_{Y \mid X}$, then knowledge of the conditional distributions alone is often sufficient determine $P_{XY}$. See Arnold and Press, 1989.

$\endgroup$
  • $\begingroup$ A little more explanation with proper references is required especially when you are contradicting answers given by others. $\endgroup$ – Shailesh Apr 17 '16 at 9:45
1
$\begingroup$

Well, I guess you have to know at least one density function among all the random variables $X_i$. Recall the basic property of conditional distributions that $P(X_i\in \mathrm{d}t_i, X_j\in \mathrm{d}t_j) = P(X_i\in \mathrm{d}t_i) P(X_j\in \mathrm{d}t_j| X_i\in \mathrm{d}t_i)$. And by induction you can figure out the joint distributions based on conditional distributions among the variables. Hope this helps!

$\endgroup$
  • $\begingroup$ No, see comment on main. $\endgroup$ – Did Apr 30 '16 at 13:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.