1
$\begingroup$

I'm trying to understand how to interpret the following which refers to determination of the mean and variance of a distribution that's the result of adding two normally distributed random variables.

$\sum_{i=1}^{n}X_i \sim N(\sum_{i=1}^{n}\mu_i, \sum_{i=1}^{n}\sigma^{2}_i)$

Let's imagine I've a bag of 100 lotto balls each with some number printed on the side, and the values of these numbers are distributed normally about the mean 1 with a variance of 1.

(OK, so normal lotto balls are integers, but let's assume a continuous notion of 'value' here)

Now, if I add another bag of 100 lotto balls, which are also normally distributed about the mean, this time of 100, with a variance of 1.

What is my expected mean and variance of the new population?

If I start from scratch and calculate the new mean, it's clear that the mean is now $101/2 = 50.5$, but from the addition of distribution formula above, I'd expect the new mean to be $\sum_{i=1}^n\mu_{i}$ which would, in this case be 101.

Similarly, the variance of such a distribution would not be 2 (i.e. $1^2 + 1^2$ ) but something much larger.

So my question here is, how am I misinterpreting this formula?

Is there a more appropriate function that will provide me the new mean and variation of a random variable that's the result of adding or mixing two known populations, calculable from totals, means, variances or other descriptive data, without resorting to recalculation from the elemental data?

And part two of my question is, under what circumstances would you use the distribution addition formula above? I can see how by adding heights of (say) paired elements I might find the results described, but how does this work if the distributions describe populations of different sizes?

$\endgroup$
4
  • $\begingroup$ Why do you think that "it's clear that the mean is now 101/2=55"? $\endgroup$ Commented Feb 8, 2016 at 15:50
  • $\begingroup$ Sorry, a typo - my reasoning here is that I've one population of 100 objects, who's mean is 1. And another population of another 100 objects, who's mean is 100. If I were to add these two populations together, I'd have 200 objects 100 of which have an effective value of 1, and another 100 objects with an effective value of 100. $\frac{(100*1)+(100*100)}{100+100}=50.5$ $\endgroup$ Commented Feb 8, 2016 at 16:18
  • 1
    $\begingroup$ But your first 100 balls have labels with mean zero. $\endgroup$ Commented Feb 8, 2016 at 16:21
  • 1
    $\begingroup$ Aha! Oops yes - sorry, let me correct that - and now I also notice, the second population's means aren't aligned - let's assume the 1st 100 are distributed around a mean of 1, and the 2nd 100 about a mean of 100. $\endgroup$ Commented Feb 8, 2016 at 16:50

1 Answer 1

1
$\begingroup$

Your question (and to an extent also the Answer by @ZoranLoncarevic) fails to distinguish explicitly between the ideas of 'adding' random variables and 'mixing' their distributions. If I add $X \sim Norm(\mu=0, \sigma^2 = 4)$ and $Y \sim Norm(\mu=8, \sigma^2 = 4)$, then I get the random variable $T = X + Y \sim Norm(\mu = 8, \sigma^2 = 8)$. If I make a 50-50 mixture of these two distributions, I get a bimodal distribution with mean 4. In the latter case the PDF of the mixture is an average of the PDFs of $X$ and $Y$. (The mixture is bimodal if the means are far enough apart; here more than twice the common SD $\sigma = 2.$)

The two histograms below illustrate the distinction. At the end of the program, approximate means, variances, and SDs of the two simulated distributions are shown.

 x = rnorm(10^5, 0, 2);  y = rnorm(10^5, 8, 2)  # SDs are 2
 t = x + y;  # sum
 h = rbinom(10^5, 1, 1/2); v = h*x + (1-h)*y    # 50-50 mixture
 par(mfrow=c(1,2))
 hist(t, br=20, prob=T, col="wheat", main="Sum of Two Normals")
   curve(dnorm(x, 8, sqrt(8)), lwd=2, col="blue", add=T)
 hist(v, prob=T, col="wheat", main="Mixture of Two Normals")
   curve(.5*dnorm(x, 0, 2) + .5*dnorm(x, 8, 2), lwd=2, col="darkgreen", add=T)
 par(mfrow=c(1,1))

 mean(t);  var(t);  sd(t)
 ## 7.991537  # Approx E(T) = 8
 ## 8.026132  # Approx Var(T) = 8
 ## 2.833043  # Approx SD(T)
 mean(v);  var(v);  sd(v)
 ## 3.988405  # Approx E(V) = 4
 ## 19.99138  # Approx Var(V)
 ## 4.471172  # Approx SD(V)

enter image description here

$\endgroup$
5
  • $\begingroup$ Yes, excellent - this is what I was looking for - I had been getting quite confused by imagining that adding two distributions was like adding two waves - but yes, mixing is the thing... One thought however; Say instead of 2 distributions to mix, I had $\{D_1, D_2, D_3, ... , D_n\}$ and the means and variances of this set of distributions were distributed normally - what might the resulting distribution look like? My guess is that it too would be normally distributed - are there methods for estimating this ultimate distribution, and what might they look like? $\endgroup$ Commented Feb 9, 2016 at 8:57
  • $\begingroup$ Adding the $D_i$'s or mixing their distributions? If adding independent $D_i$,then normal sum with sum of means for the mean and sum of variances for the variance. If mixing, then average the PDFs in appropriate proportions. $\endgroup$
    – BruceET
    Commented Feb 9, 2016 at 18:07
  • $\begingroup$ I suppose the line I'm thinking along is that how after multiple mixings the process should apply as a candidate for the central limit theorem, and that, over time and given a large enough sample size $n$ in ${D_1,D_2,D_3, ... ,D_n}$, the distribution of the means of distributions $\bar{d}$ for $D$ should be Normal. $\endgroup$ Commented Feb 10, 2016 at 10:20
  • 1
    $\begingroup$ Provided that the means are not too diverse, a mixture of normals is nearly normal (but not because of the CLT). For example, heights of people are often taken as normal, but the population is a mixture of males and females (known to have avg hts differing by an inch or two) and of different ethnic groups (ditto). The example in the right-hand graph above has 2 means differing by more than twice the common SD, so it is distinctly bimodal. A small sample of a 50;50 mix of Norm(100,15) and Norm(105,15) would not be normal, but hard to distinguish from normal. $\endgroup$
    – BruceET
    Commented Feb 10, 2016 at 17:57
  • 1
    $\begingroup$ As for your second point: CLT would tend to make $sample\; means$ from a mixture distribution (or any other dist'n having a mean and variance) converge to normal for large sample sizes. $\endgroup$
    – BruceET
    Commented Feb 10, 2016 at 18:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .