I can intuitively understand that as I take more samples from a random variable $X$ (gaussian distribution), the mean would approach $E(X)$.

But what I don't get is if I look at it mathematically. Say I took two samples $x_1$, $x_2$. The mean would be $(x_1+x_2)/2$. But variances add, which is contrary to the previous idea that the variance for the mean should get smaller as I increase the number of samples.

I've asked this question elsewhere and from what I understand variances in this case do not add. Somebody talked about there being a difference between "i.i.d. and independence". But I don't get it, either something is independent or it is not. The way I see it acting like $x_1$ and $x_2$ are dependent would be committing the gamblers fallacy.

Or is there a difference between a) the process of first sampling $x_1$, then $x_2$ and b) viewing the sampling of $x_1$ and $x_2$ as a single process?

I tried following this proof this for the sum of variances and using twice the same random variable $X$ but that resulted in nonsense.

In short: What's the deal with independence in this context and how does it influence $var(x_1 +x_2)$?

  • $\begingroup$ "Variances add". So the variance of $X_1+X_2$ is the sum of the variances. But we're not talking about $X_1+X_2$, we're talking about $(X_1+X_2)/2$. $\endgroup$ – David C. Ullrich Feb 9 '16 at 13:37

$V(aX)=a^2V(X)$. Hence, if $X_1,\dots X_n$ are i.i.d.: $$V\left(\frac{1}{n}\sum_{i=1}^nX_i\right)=\frac{1}{n^2}V\left(\sum_{i=1}^nX_i\right)=\frac{1}{n^2}nV(X_1)=\frac{V(X_1)}{n}$$

So the variance of the sample mean is indeed decreasing as the sample size increases.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.