I have noticed that sometimes mixtures of normal distributions can be approximated very well by a simple normal distribution. For instance, the mixture of the normal distributions $\mathcal{N}(0,1)$ and $\mathcal{N}(1,1)$ (with both distributions having a weight $w_i$ of 0.5) is well approximated by $\mathcal{N}(0.5,1.2544)$; the density functions of both distributions differ by about 0.004 or less over the whole continuum. My question is, under which conditions is such a normal approximation of a mixture of normal distributions possible?

  • $\begingroup$ It is always possible, since the mixture will have a mean and variance ($\frac12$ and $\frac54$ in your example). The real question is when is such an approximation good for a suitable definition of good $\endgroup$ – Henry Nov 16 '17 at 19:16
  • $\begingroup$ A 50:50 mixture of two normal distributions for which the absolute difference between the two means is less than twice the common SD is unimodal. (See the Examples section of the Wikipedia article on 'mixture distributions'.) Roughly speaking, normal approximations to mixture distributions work best when the mixture is unimodal (for central values) and the components have a common variance (for values in tails). $\endgroup$ – BruceET Nov 16 '17 at 22:32

Continued Comment: Single samples of size $n = 1000$ from each of three mixture distributions illustrate three possibilities:

$U$ is a 50:50 mixture of $\mathsf{Norm}(10, 2)$ and $\mathsf{Norm}(12, 2).$ According to a Shapiro-Wilk test, my sample is consistent with normal: P-value = 0.134.

$V$ is a 50:50 mixture of $\mathsf{Norm}(10, 2)$ and $\mathsf{Norm}(16, 2).$ Judged clearly not normal: $\;\;$ Shapiro-Wilk P-value < 0.0005.

$W$ is a 50:50 mixture of $\mathsf{Norm}(10, 2)$ and $\mathsf{Norm}(13, 6).$ Judged clearly not normal: $\;\;$Shapiro-Wilk P-value < 0.0005.

Different samples from these distributions may give somewhat different results, but the idea survives: mixtures of distributions with approximately equal means and equal variances are better approximations to normal.

The graphs below show my three samples along with the density functions of the three mixture distributions.

enter image description here

Note: Heights of humans are often modeled as approximately normal. They are close to a 50:50 mixture of women and men, whose average heights differ by perhaps 3-4 inches, which is smaller than two standard deviations. Heights might be more closely modeled as normal by looking at women and men separately.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.