2
$\begingroup$

Let $X_1$ and $X_2$ be two Bernoulli random variables, with $P(X_1=1)=p_1$ and $P(X_2=1)=p_2$. The following discussion showed how to generate a pair of correlated Bernoulli variables with correlation coefficient $\rho$. I want to do the same but for 3 variables $X_1$, $X_2$, $X_3$ with respective probability $p_1$, $p_2$ and $p_3$ and identical correlation coefficient $\rho>0$ between $X_1$ and $X_2$, $X_1$ and $X_3$ and $X_2$ and $X_3$. Intuitively, it seems that this has multiple solutions depending on the higher order association between $X_1$, $X_2$, $X_3$, which I would then fix to 0. Is there a way to generate such distributions of Bernoulli variables ? By "generate", I don't mean produce with the help of an algorithm that would converge towards the desired distribution, but I mean more how we can construct it, i.e. calculate the probability of each combinations of the three Bernoulli variables. If yes, can it be generalized to more than 3 variables (assuming null association between more than two variables)?

$\endgroup$
4
  • $\begingroup$ What is your measure of "the higher order association between $X_1,X_2,X_3$"? $\endgroup$
    – kccu
    Commented Jan 29, 2020 at 14:54
  • $\begingroup$ Also, based on the bounds given in this answer, it is only possible to have identical correlation coefficient $\rho$ between all pairs of $X_1,X_2,X_3$ if $\rho \geq -\frac{1}{2}$. $\endgroup$
    – kccu
    Commented Jan 29, 2020 at 14:56
  • $\begingroup$ @kccu: Thanks for the answer. 1) I mean correlation between 3 or more variables should be 0 (i.e. $E(X_1 X_2 X_3)=E(X_1) E(X_2) E(X_3)$). Is it correct so say that? 2) I only want to apply it to positive $\rho$ so should not be a problem. $\endgroup$ Commented Jan 29, 2020 at 15:03
  • $\begingroup$ I've never seen that (or any) notion of correlation between 3 random variables, which is why I ask. See here. $\endgroup$
    – kccu
    Commented Jan 29, 2020 at 15:13

1 Answer 1

3
$\begingroup$

To construct a measurable space let $\Omega=\left\{ 0,1\right\} ^{3}$ be equipped with its powerset as $\sigma$-algebra.

It is handsome to think of an outcome $\omega\in\Omega$ as a function $\left\{ 1,2,3\right\} \to\left\{ 0,1\right\} $.

For $i=1,2,3$ let $X_{i}:\Omega\to\mathbb{R}$ be prescribed by $\omega\mapsto\omega\left(i\right)$.

The outcome space has $2^{3}=8$ elements so that the $\sigma$-algebra contains $2^8$ events and we must find a probability measure $P$ on it that satisfies:

  • $p_{1}=P\left(X_{1}^{-1}\left(\left\{ 1\right\} \right)\right)$

  • $p_{2}=P\left(X_{2}^{-1}\left(\left\{ 1\right\} \right)\right)$

  • $p_{3}=P\left(X_{3}^{-1}\left(\left\{ 1\right\} \right)\right)$

  • $p_{1}p_{2}p_{3}=P\left(X_{1}^{-1}\left(\left\{ 1\right\} \right)\cap X_{2}^{-1}\left(\left\{ 1\right\} \right)\cap X_{3}^{-1}\left(\left\{ 1\right\} \right)\right)$

  • $\rho\sqrt{p_{1}\left(1-p_{1}\right)p_{2}\left(1-p_{2}\right)}+p_{1}p_{2}=P\left(X_{1}^{-1}\left(\left\{ 1\right\} \right)\cap X_{2}^{-1}\left(\left\{ 1\right\} \right)\right)$

  • $\rho\sqrt{p_{1}\left(1-p_{1}\right)p_{3}\left(1-p_{3}\right)}+p_{1}p_{3}=P\left(X_{1}^{-1}\left(\left\{ 1\right\} \right)\cap X_{3}^{-1}\left(\left\{ 1\right\} \right)\right)$

  • $\rho\sqrt{p_{2}\left(1-p_{2}\right)p_{3}\left(1-p_{3}\right)}+p_{2}p_{3}=P\left(X_{2}^{-1}\left(\left\{ 1\right\} \right)\cap X_{3}^{-1}\left(\left\{ 1\right\} \right)\right)$

where the last $3$ equalities are based on the equality: $$\rho\left(X_{i},X_{j}\right)\sqrt{p_{i}\left(1-p_{i}\right)p_{j}\left(1-p_{j}\right)}=\rho\left(X_{i},X_{j}\right)\sigma_{X_{i}}\sigma_{X_{j}}=\mathsf{Cov}\left(X_{i},X_{j}\right)=\mathbb{E}X_{i}X_{j}-\mathbb{E}X_{i}\mathbb{E}X_{j}=$$$$\mathbb{E}X_{i}X_{j}-p_{i}p_{j}$$

There are $8$ disjoint sets of shape $X_{1}^{-1}\left(\left\{ x\right\} \right)\cap X_{2}^{-1}\left(\left\{ y\right\} \right)\cap X_{3}^{-1}\left(\left\{ z\right\} \right)$ involved (where $x,y,z\in\{0,1\}$) that cover the outcome space and each of them has a probability. Giving these events a probability comes actually to the same as determining the probability measure.

Denoting these probabilities as $a,u,v,w,r,s,t,z$ we meet the following conditions:

  • $1=a+u+v+w+r+s+t+z$

  • $p_{1}=a+u+v+s=$

  • $p_{2}=a+u+w+r$

  • $p_{3}=a+v+w+t$

  • $p_{1}p_{2}p_{3}=a$

  • $\rho\sqrt{p_{1}\left(1-p_{1}\right)p_{2}\left(1-p_{2}\right)}+p_{1}p_{2}=a+u$

  • $\rho\sqrt{p_{1}\left(1-p_{1}\right)p_{3}\left(1-p_{3}\right)}+p_{1}p_{3}=a+v$

  • $\rho\sqrt{p_{2}\left(1-p_{2}\right)p_{3}\left(1-p_{3}\right)}+p_{2}p_{3}=a+w$

Here e.g. $a$ stands for the probability of $X_{1}^{-1}\left(\left\{ 1\right\} \right)\cap X_{2}^{-1}\left(\left\{ 1\right\} \right)\cap X_{3}^{-1}\left(\left\{ 1\right\} \right)$ and $u$ for the probability of $X_{1}^{-1}\left(\left\{ 1\right\} \right)\cap X_{2}^{-1}\left(\left\{ 1\right\} \right)\cap X_{3}^{-1}\left(\left\{ 0\right\} \right)$ and $s$ for the probability of $X_{1}^{-1}\left(\left\{ 1\right\} \right)\cap X_{2}^{-1}\left(\left\{ 0\right\} \right)\cap X_{3}^{-1}\left(\left\{ 0\right\} \right)$.

So we meet $8$ equalities on $8$ unknown variables. This however is not a guarantee that there is a proper solution because there are also constraints on base of inequalities. E.g. the probabilities cannot be negative.

$\endgroup$
1
  • 1
    $\begingroup$ Out of upvotes for the day but +1 for the detailed answer. $\endgroup$
    – Math1000
    Commented Jan 30, 2020 at 21:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .