2
$\begingroup$

I am trying to understand whether it is possible and if so then how to find a joint distribution given marginals and a correlation matrix.

In particular, suppose $X_{1},\ldots,X_{n}$ are discrete random variables. I am given their marginals $p_{X_{1}},\ldots,p_{X_{n}}$ and their correlation matrix. Can I construct (uniquely?) the joint $p_{X_{1},\ldots,X_{n}}$?

I tried this for $n=2$, assuming each of the two random variables has two possible outcomes $a$ and $b>a$, and assuming that the marginals of $X_{1}$ and $X_{2}$ are the same given by $\mathbb{P}[X_{1}=a]=c$. Let $r$ be the correlation of $X_{1}$ and $X_{2}$. My Mathematica readily calculates unique joint distribution provided $c(1-r)<1$ and claims there is no joint otherwise.

How would I generalize to more random variables with more than two possible outcomes? What I do in Mathematica (calculate correlation from joint distribution and then solve for the joint distribution that gives $r$) does not generalize nicely. Also, are there some results that would tell me what I am doing can/cannot be done? Uniquely?

I tried reading StackExchange questions/answers here, here, here, here, and here. But none of the answers help me forward. Also, I know that this question is trivial when the random variables are independent. So let's disregard that special case.

$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

In general, the joint distribution is not uniquely determined by the marginals. The correlations only give second order moment information so are still not enough to pin down a unique joint distribution.

Adapting this example from wiki, suppose $W$ is a random variable that takes values $-1,1$ with equal probability, and $X\sim N(0,1)$. Assume $X$ is independent of $W$. Define $Y=WX$. You can show $X,Y$ are uncorrelated and are marginally standard normal but not independent. However, two iid standard normals are also uncorrelated.

Here is a discrete example with infinitely many joint distributions that yield the same uncorrelated marginal distributions:

Suppose $X\in \{-1,0,1\}$ and $Y\in \{0,1\}$. For some $p\in (1/6,1/3)$, consider the joint distribution $$\begin{align} P(X=-1,Y=1)&=P(X=1,Y=1)=p,\\ P(X=-1,Y=0)&=P(X=1,Y=0)=1/3-p,\\ P(X=0,Y=0)&=2p-1/3\\ P(X=0,Y=1)&=2/3-2p\\ \end{align} $$

The marginals are given by

$$P(X=x)=1/3 \quad \forall x\in \{-1,0,1\}\\ P(Y=0)=1/3,P(Y=1)=2/3$$

and $X,Y$ are uncorrelated (note $X$ has mean zero):

$$E[XY]=(-1)(1)p+(1)(1)p=0.$$

$\endgroup$
5
  • $\begingroup$ Thank you, this is insightful. Yes, correlation pins down only the second order. But this is very heuristic. The example is beautiful. However, I do not understand how $Y$ is standard normal. In the example, $a=1$, so yes. But for general $a$, the variance of $Y$ is $a^{2}$ if I am correct? Moreover, I have not been able to find any formal (book) treatment of the issues I am asking about. And if not results, then (counter-)examples. Maybe because questions about uniqueness/existence of a joint given marginals and other conditions seems to have distribution/parameter-specific answers. $\endgroup$
    – Jan
    Mar 23 at 20:16
  • $\begingroup$ @Jan You are right! Thanks for pointing out! I have updated the post to show a less ambitious counterexample, namely that the same marginals and correlation of zero can produce at least two joint distributions; also, the wiki link has another "asymmetric example" of uncorrelated standard normals. You may also be interested in copulas. $\endgroup$
    – Golden_Ratio
    Mar 23 at 20:55
  • $\begingroup$ Thank you. I noticed copulas in some of my reading but did not pay much attention assuming that they apply to continuous r.v.s and I am mainly interested in discrete ones. I will keep educating myself. $\endgroup$
    – Jan
    Mar 23 at 21:12
  • $\begingroup$ @Jan I added a discrete example with infinitely many joint distributions--hope that helps. $\endgroup$
    – Golden_Ratio
    Mar 23 at 22:57
  • $\begingroup$ The discrete example is great, and proves that my ''quest for uniqueness'' is bound to fail. Thanks for proving me wrong. $\endgroup$
    – Jan
    Mar 24 at 13:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .