1
$\begingroup$

Say you've got $i=1:N$ observations, $X_i$. Each observation is the mean of a Gaussian with an associated variance $Var(X_i)$. The variances are NOT identical. The expectation of $X$ should simply be their mean $N^{-1}\displaystyle\sum X$. The variance of the mean should be $$ Var\left(N^{-1}\displaystyle\sum X_i\right) = N^{-2}\displaystyle\sum Var(X_i) $$ Is this correct? I'm following the basic formula on Wikipedia, but I'm getting implausibly small variances. Am I missing something? The above formula is usually used in the context of equal variances, but it doesn't say anything about requiring equal variances.

$\endgroup$
3
  • $\begingroup$ The variances need not be identical. $\endgroup$ Commented Apr 8, 2016 at 3:58
  • 1
    $\begingroup$ Your wording is not totally clear (you have not defined $X$), but if the $X_i$ are independent with each having variance $\sigma^2_i$ then the variance of $\displaystyle \sum_i X_i$ is $\displaystyle \sum_i \sigma^2_i$ and the variance of $\displaystyle \frac1n \sum_i X_i$ is $\displaystyle \frac{1}{n^2} \sum_i \sigma^2_i$, which I think is what you intended... $\endgroup$
    – Henry
    Commented Apr 8, 2016 at 8:44
  • $\begingroup$ ... If the original variances are are all equal to $\sigma^2$ then this simplifies to saying the variance of $\displaystyle \sum_i X_i$ is $n \sigma^2$ and the variance of $\displaystyle \frac1n \sum_i X_i$ is $\displaystyle \frac{\sigma^2}{n}$. Take square-roots of variances for standard deviations $\endgroup$
    – Henry
    Commented Apr 8, 2016 at 8:47

1 Answer 1

0
$\begingroup$

Yes, this is so.   The random variables do not need to be identically distributed.   However the result requires the random variables to be independent.

Let ${\{(X_i, \mu_i, \sigma_i^2)\}}_{i=1}^N$ be a series of mutually independent random variables, their means, and variances.

Let $\bar X:=\tfrac 1 N \sum\limits_{i=1}^N X_i$ be the mean of the series (ie: the sample mean).

Then indeed the expectation of the sample mean is: $\mathsf E(\bar X) = \tfrac 1 N\sum\limits_{i=1}^N \mu_i$

And the variance of the sample mean is:$$\begin{align} \mathsf{Var}(\bar X) ~=~& \mathsf {Cov}(\tfrac 1 N \sum\limits_{i=1}^N X_i,\tfrac 1 N \sum\limits_{j=1}^N X_j) \\[1ex] =~&\tfrac 1{N^2} \sum_{i=1}^N\sum_{j=1}^N \mathsf{Cov}(X_i,X_j) \\[1ex] =~& \tfrac 1{N^2} \sum_{i=1}^N \mathsf{Var}(X_i) & ^\dagger \\[1ex] =~& \tfrac {1}{N^2} \sum_{i=1}^N \sigma_i^2 \end{align}$$

(† because $\forall i\forall j: \big[i\neq j ~\to~ \mathsf{Cov}(X_i,X_j)=0\big]$ due to independence.)

And indeed, as the count $N$ increases infinitely the variance of the sample mean will decrease vanishingly.   $~\lim\limits_{N\to\infty} \mathsf{Var}(\bar X)=0$


PS: This is not the mean of the sample variance, if that is what is confusing you.

$\endgroup$
2
  • $\begingroup$ Thanks for confirming my intuition. I wonder if the variances that I'm getting are so small because the quantities that I'm dealing with aren't in fact independent... $\endgroup$
    – user329657
    Commented Apr 8, 2016 at 20:25
  • $\begingroup$ What if the variances are not bounded? That is, I know $\mathbb{V}\text{ar}(\bar{X}) \to 0$ if $\mathbb{V}\text{ar}(X_i) < M$ for all $i \in \mathbb{N}$, but if the variance are unbounded does this result hold? $\endgroup$ Commented Nov 22, 2021 at 3:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .