# How to generate correlated Bernoulli variables?

Let $$X_1$$ and $$X_2$$ be two Bernoulli random variables, with $$P(X_1=1)=p_1$$ and $$P(X_2=1)=p_2$$. The following discussion showed how to generate a pair of correlated Bernoulli variables with correlation coefficient $$\rho$$. I want to do the same but for 3 variables $$X_1$$, $$X_2$$, $$X_3$$ with respective probability $$p_1$$, $$p_2$$ and $$p_3$$ and identical correlation coefficient $$\rho>0$$ between $$X_1$$ and $$X_2$$, $$X_1$$ and $$X_3$$ and $$X_2$$ and $$X_3$$. Intuitively, it seems that this has multiple solutions depending on the higher order association between $$X_1$$, $$X_2$$, $$X_3$$, which I would then fix to 0. Is there a way to generate such distributions of Bernoulli variables ? By "generate", I don't mean produce with the help of an algorithm that would converge towards the desired distribution, but I mean more how we can construct it, i.e. calculate the probability of each combinations of the three Bernoulli variables. If yes, can it be generalized to more than 3 variables (assuming null association between more than two variables)?

• What is your measure of "the higher order association between $X_1,X_2,X_3$"?
– kccu
Commented Jan 29, 2020 at 14:54
• Also, based on the bounds given in this answer, it is only possible to have identical correlation coefficient $\rho$ between all pairs of $X_1,X_2,X_3$ if $\rho \geq -\frac{1}{2}$.
– kccu
Commented Jan 29, 2020 at 14:56
• @kccu: Thanks for the answer. 1) I mean correlation between 3 or more variables should be 0 (i.e. $E(X_1 X_2 X_3)=E(X_1) E(X_2) E(X_3)$). Is it correct so say that? 2) I only want to apply it to positive $\rho$ so should not be a problem. Commented Jan 29, 2020 at 15:03
• I've never seen that (or any) notion of correlation between 3 random variables, which is why I ask. See here.
– kccu
Commented Jan 29, 2020 at 15:13

To construct a measurable space let $$\Omega=\left\{ 0,1\right\} ^{3}$$ be equipped with its powerset as $$\sigma$$-algebra.

It is handsome to think of an outcome $$\omega\in\Omega$$ as a function $$\left\{ 1,2,3\right\} \to\left\{ 0,1\right\}$$.

For $$i=1,2,3$$ let $$X_{i}:\Omega\to\mathbb{R}$$ be prescribed by $$\omega\mapsto\omega\left(i\right)$$.

The outcome space has $$2^{3}=8$$ elements so that the $$\sigma$$-algebra contains $$2^8$$ events and we must find a probability measure $$P$$ on it that satisfies:

• $$p_{1}=P\left(X_{1}^{-1}\left(\left\{ 1\right\} \right)\right)$$

• $$p_{2}=P\left(X_{2}^{-1}\left(\left\{ 1\right\} \right)\right)$$

• $$p_{3}=P\left(X_{3}^{-1}\left(\left\{ 1\right\} \right)\right)$$

• $$p_{1}p_{2}p_{3}=P\left(X_{1}^{-1}\left(\left\{ 1\right\} \right)\cap X_{2}^{-1}\left(\left\{ 1\right\} \right)\cap X_{3}^{-1}\left(\left\{ 1\right\} \right)\right)$$

• $$\rho\sqrt{p_{1}\left(1-p_{1}\right)p_{2}\left(1-p_{2}\right)}+p_{1}p_{2}=P\left(X_{1}^{-1}\left(\left\{ 1\right\} \right)\cap X_{2}^{-1}\left(\left\{ 1\right\} \right)\right)$$

• $$\rho\sqrt{p_{1}\left(1-p_{1}\right)p_{3}\left(1-p_{3}\right)}+p_{1}p_{3}=P\left(X_{1}^{-1}\left(\left\{ 1\right\} \right)\cap X_{3}^{-1}\left(\left\{ 1\right\} \right)\right)$$

• $$\rho\sqrt{p_{2}\left(1-p_{2}\right)p_{3}\left(1-p_{3}\right)}+p_{2}p_{3}=P\left(X_{2}^{-1}\left(\left\{ 1\right\} \right)\cap X_{3}^{-1}\left(\left\{ 1\right\} \right)\right)$$

where the last $$3$$ equalities are based on the equality: $$\rho\left(X_{i},X_{j}\right)\sqrt{p_{i}\left(1-p_{i}\right)p_{j}\left(1-p_{j}\right)}=\rho\left(X_{i},X_{j}\right)\sigma_{X_{i}}\sigma_{X_{j}}=\mathsf{Cov}\left(X_{i},X_{j}\right)=\mathbb{E}X_{i}X_{j}-\mathbb{E}X_{i}\mathbb{E}X_{j}=$$$$\mathbb{E}X_{i}X_{j}-p_{i}p_{j}$$

There are $$8$$ disjoint sets of shape $$X_{1}^{-1}\left(\left\{ x\right\} \right)\cap X_{2}^{-1}\left(\left\{ y\right\} \right)\cap X_{3}^{-1}\left(\left\{ z\right\} \right)$$ involved (where $$x,y,z\in\{0,1\}$$) that cover the outcome space and each of them has a probability. Giving these events a probability comes actually to the same as determining the probability measure.

Denoting these probabilities as $$a,u,v,w,r,s,t,z$$ we meet the following conditions:

• $$1=a+u+v+w+r+s+t+z$$

• $$p_{1}=a+u+v+s=$$

• $$p_{2}=a+u+w+r$$

• $$p_{3}=a+v+w+t$$

• $$p_{1}p_{2}p_{3}=a$$

• $$\rho\sqrt{p_{1}\left(1-p_{1}\right)p_{2}\left(1-p_{2}\right)}+p_{1}p_{2}=a+u$$

• $$\rho\sqrt{p_{1}\left(1-p_{1}\right)p_{3}\left(1-p_{3}\right)}+p_{1}p_{3}=a+v$$

• $$\rho\sqrt{p_{2}\left(1-p_{2}\right)p_{3}\left(1-p_{3}\right)}+p_{2}p_{3}=a+w$$

Here e.g. $$a$$ stands for the probability of $$X_{1}^{-1}\left(\left\{ 1\right\} \right)\cap X_{2}^{-1}\left(\left\{ 1\right\} \right)\cap X_{3}^{-1}\left(\left\{ 1\right\} \right)$$ and $$u$$ for the probability of $$X_{1}^{-1}\left(\left\{ 1\right\} \right)\cap X_{2}^{-1}\left(\left\{ 1\right\} \right)\cap X_{3}^{-1}\left(\left\{ 0\right\} \right)$$ and $$s$$ for the probability of $$X_{1}^{-1}\left(\left\{ 1\right\} \right)\cap X_{2}^{-1}\left(\left\{ 0\right\} \right)\cap X_{3}^{-1}\left(\left\{ 0\right\} \right)$$.

So we meet $$8$$ equalities on $$8$$ unknown variables. This however is not a guarantee that there is a proper solution because there are also constraints on base of inequalities. E.g. the probabilities cannot be negative.

• Out of upvotes for the day but +1 for the detailed answer. Commented Jan 30, 2020 at 21:51