# Variance of the mean of independent (not IID) normal random variables

Say you've got $i=1:N$ observations, $X_i$. Each observation is the mean of a Gaussian with an associated variance $Var(X_i)$. The variances are NOT identical. The expectation of $X$ should simply be their mean $N^{-1}\displaystyle\sum X$. The variance of the mean should be $$Var\left(N^{-1}\displaystyle\sum X_i\right) = N^{-2}\displaystyle\sum Var(X_i)$$ Is this correct? I'm following the basic formula on Wikipedia, but I'm getting implausibly small variances. Am I missing something? The above formula is usually used in the context of equal variances, but it doesn't say anything about requiring equal variances.

• The variances need not be identical. Commented Apr 8, 2016 at 3:58
• Your wording is not totally clear (you have not defined $X$), but if the $X_i$ are independent with each having variance $\sigma^2_i$ then the variance of $\displaystyle \sum_i X_i$ is $\displaystyle \sum_i \sigma^2_i$ and the variance of $\displaystyle \frac1n \sum_i X_i$ is $\displaystyle \frac{1}{n^2} \sum_i \sigma^2_i$, which I think is what you intended... Commented Apr 8, 2016 at 8:44
• ... If the original variances are are all equal to $\sigma^2$ then this simplifies to saying the variance of $\displaystyle \sum_i X_i$ is $n \sigma^2$ and the variance of $\displaystyle \frac1n \sum_i X_i$ is $\displaystyle \frac{\sigma^2}{n}$. Take square-roots of variances for standard deviations Commented Apr 8, 2016 at 8:47

Yes, this is so.   The random variables do not need to be identically distributed.   However the result requires the random variables to be independent.

Let ${\{(X_i, \mu_i, \sigma_i^2)\}}_{i=1}^N$ be a series of mutually independent random variables, their means, and variances.

Let $\bar X:=\tfrac 1 N \sum\limits_{i=1}^N X_i$ be the mean of the series (ie: the sample mean).

Then indeed the expectation of the sample mean is: $\mathsf E(\bar X) = \tfrac 1 N\sum\limits_{i=1}^N \mu_i$

And the variance of the sample mean is:\begin{align} \mathsf{Var}(\bar X) ~=~& \mathsf {Cov}(\tfrac 1 N \sum\limits_{i=1}^N X_i,\tfrac 1 N \sum\limits_{j=1}^N X_j) \\[1ex] =~&\tfrac 1{N^2} \sum_{i=1}^N\sum_{j=1}^N \mathsf{Cov}(X_i,X_j) \\[1ex] =~& \tfrac 1{N^2} \sum_{i=1}^N \mathsf{Var}(X_i) & ^\dagger \\[1ex] =~& \tfrac {1}{N^2} \sum_{i=1}^N \sigma_i^2 \end{align}

(† because $\forall i\forall j: \big[i\neq j ~\to~ \mathsf{Cov}(X_i,X_j)=0\big]$ due to independence.)

And indeed, as the count $N$ increases infinitely the variance of the sample mean will decrease vanishingly.   $~\lim\limits_{N\to\infty} \mathsf{Var}(\bar X)=0$

PS: This is not the mean of the sample variance, if that is what is confusing you.

• Thanks for confirming my intuition. I wonder if the variances that I'm getting are so small because the quantities that I'm dealing with aren't in fact independent... Commented Apr 8, 2016 at 20:25
• What if the variances are not bounded? That is, I know $\mathbb{V}\text{ar}(\bar{X}) \to 0$ if $\mathbb{V}\text{ar}(X_i) < M$ for all $i \in \mathbb{N}$, but if the variance are unbounded does this result hold? Commented Nov 22, 2021 at 3:27