# Mixture of densities vs mixture of random variables

Let $$f(x),g(x),h(x)$$ be probability density functions. Assume that $$f$$ is a (nontrivial) mixture of $$g$$ and $$h$$, that is, there is some $$p\in (0,1)$$, such that $$f(x)=pg(x)+(1-p)h(x)\hspace{20mm} (*)$$ holds. Let $$X$$ be a random variable with density $$f(x)$$.

Question: Does ($$*$$) imply that there are always two random variables $$Y,Z$$ with densities $$g(x),h(x)$$, and a 0-1 valued random variable $$\xi$$, independent of $$Y,Z$$, with $$\Pr(\xi=1)=p$$, $$\Pr(\xi=0)=1-p$$, such that $$\Pr{\Large (}X=\xi Y+(1-\xi)Z{\Large )}\;=\;1\hspace{20mm} (**)$$ holds? In other words, is it true that if the density of $$X$$ is a mixture of two other densities, then $$X$$ can always be realized as mixture of two other random variables with the respective densities, in the sense of ($$**$$)?

Note: If we just take any $$Y,Z,\xi$$ with the above properties, then the random variable $$\widetilde X=\xi Y+(1-\xi)Z$$ clearly has distribution $$f(x)=pg(x)+(1-p)h(x)$$, as $$X$$. However, nothing guarantees that $$\Pr(\widetilde X=X)=1$$. How to choose $$Y,Z,\xi\;$$ if we actually want to realize $$X$$, not just any other random variable with the same distribution?

## 1 Answer

You are right that $$\widetilde{X}$$ and $$X$$ have the same distribution $$f$$ but they are not necessary equal. That's what i.i.d. variables are all about.

To your original question: I suppose it depends on your probability space, what qualifies as events, what qualifies as r.v., etc. I am not a theorist so I cannot answer that rigorously. As a practical matter, a density of form $$f$$ would usually arise because the author has some underlying $$Y, Z, \xi$$ in mind. But as a theoretical matter, whether you can retroactively construct $$Y,Z,\xi$$... On the same probability space, the answer IMHO is No. On some expanded space (in a sense)... that I don't know.

Here's a discrete example, so we're dealing with pmf as opposed to pdf... hope such a change is OK with you? Consider a fair coin flip, so $$P(X=0) = P(X=1) = 1/2$$, i.e. its pmf is the vector $$[\frac12, \frac12]$$. This is a mixture:

$$[\frac12, \frac12] = \frac12 \times [\frac13, \frac23] + \frac12 \times [\frac23, \frac13]$$

However, the original probability space has just two sample points $$\Omega = \{Heads, Tails\}$$, and using those alone you cannot define any r.v. (in the sense of a function $$\Omega \to \mathbb{R}$$) with pmf $$[\frac13, \frac23]$$, so in that sense the r.v. $$Y$$ where $$P(Y=0) = 1/3, P(Y=1)=2/3$$ doesn't exist, and $$X$$ cannot be "realized" as a mix of $$Y$$ and $$Z$$ if we restrict to the same space. Now of course you can invent a new probability space, with two biased coins, to implement all of $$Y, Z, \xi$$.

• In the second counterexample it is not clear why an r.v. $Y$ with pmf [1/3,2/3] cannot exist. One can take a 0-1 valued r.v. $Y$ with $\Pr(Y=0)=1/3$ and $\Pr(Y=1)=2/3$. Commented Sep 29, 2019 at 18:29
• The counterexample to the Note is wrong, $\widetilde X$ will not be Gaussian. It is not a weighted sum of $Y$ and $Z$, it is their mixture. It means, flip a (possibly biased) coin, and pick one of $Y, Z$ depending on the outcome. Such a mixture may very well be bimodal, not Gaussian. Commented Sep 29, 2019 at 20:39
• My apologies re: the Note. I got confused between $\xi$ and $p$... :( I have deleted that part of my answer. Re: the non-existence of $Y$ in the coin example, I am merely making the point that you cannot find such an r.v. in the same probability space. Obviously you can find such an r.v. if you expand the probability space (with biased coins). Commented Sep 29, 2019 at 21:37