This is the question:

For i.i.d. r.v.s $X_1, . . . ,X_n$ with mean $μ$ and variance $\sigma^2$, give a value of $n$ (as a specific number) that will ensure that there is at least a $99\%$ chance that the sample mean will be within $2$ standard deviations of the true mean $μ$.

This is the solution:

We have to find $n$ such that

$$P(|\overline{X}_n-\mu|>2\mu) \leq 0.01$$

By Chebyshev's inequality (in the form $P(|Y-EY|>c) \leq \frac{\text{Var}(Y)}{c^2}$), we have

$$P(|\overline{X}_n-\mu|>2\mu) \leq \frac{\text{Var}(\overline{X}_n)}{(2\sigma)^2} = \frac{\frac{\sigma^2}{n}}{4\sigma^2}=\frac{1}{4n}$$

So the desired inequality holds if $n \geq 25$.

So what I'm confused about is why $\text{Var}(\overline{X}_n)=\frac{\sigma^2}{n}$. What does it mean to take the variance of the sample mean? I did some research and it seems like sample variance (is this even called sample variance?) should be divided $n-1$ and not $n$ as seen in the solution. Unfortunately the course I'm taking has not gone over any of this. I'm hoping someone could point me in the right direction or even providing the necessary terms to search to understand this more.

  • $\begingroup$ How the denominator becomes $4\sigma^2$, shouldn't it be $4\mu^2$ instead? $\endgroup$ Nov 9 '16 at 1:06
  • $\begingroup$ Ok, I think the first expression should be $P(|\overline{X}_n-\mu|>2\sigma) \leq \frac{\text{Var}(\overline{X}_n)}{(2\sigma)^2} = \frac{\frac{\sigma^2}{n}}{4\sigma^2}=\frac{1}{4n}$ instead? $\endgroup$ Nov 9 '16 at 1:25
  • $\begingroup$ and why should the inequality not hold if n is 24 for instance? $\endgroup$ Nov 9 '16 at 1:26

$\newcommand{\var}{\operatorname{Var}}$ We have \begin{align*} \var(\bar X_n) &= \var\left(\frac{X_1+\dotsb+X_n}{n}\right)\\ &=\frac{1}{n^2}\left[\var(X_1)+\dotsb+\var(X_n)\right]\tag 1\\ &=\frac{n}{n^2}\var(X_1)\\ &=\frac{\sigma^2}{n} \end{align*} where $(1)$ is true by the implicit assumption that the $X_i$ are independent.

When we have data, and we need to estimate the variance of the population, then we use the formula $$s^2 = \frac{1}{n-1}\sum_{k = 1}^n(x_k-\bar x).$$ This gives an unbiased estimate of the population variance.

  • $\begingroup$ Is there an intuitive way to think about the result $\frac{\sigma^2}{n}$. I would have thought that the variance of the sample mean would just be the variance of one of the random variables $X$. Is it because as we have more samples, ie. $n$ is large, we expect it to vary less as we are converging to the actual variance of the data? $\endgroup$
    – jlcv
    Jun 3 '16 at 20:25
  • 1
    $\begingroup$ I think one intuitive answer would be to say that we are looking at the mean, and hence we divide by $n$. It doesn't make (to me) sense to say that the variance of the sample mean is just the variance of $X$. To the second part, no. I believe that as $n$ increases the variance of the mean should approach $0$. My reasoning is that at the extreme end, if we sample the entire population, then the mean should not vary. The mean of the sample is actually the true population mean when we sample the whole thing. $\endgroup$
    – Em.
    Jun 3 '16 at 20:33
  • $\begingroup$ I guess also by the work that I have shown, $\lim\limits_{n\to\infty} \var(\bar X_n) = 0$. But I'm not super 100 on my stats. I could be making a mistake. Hehe. $\endgroup$
    – Em.
    Jun 3 '16 at 20:41
  • $\begingroup$ @ZoomBee: Assuming independence, $\var(\sum X_k ) = \sum \var(X_k) = n \sigma^2 $ so $\var(\bar{X} )=\var(\frac1n \sum X_k ) = \frac1{n^2} n \sigma^2= \frac{\sigma^2 }{n} $. The convergence, by the law of large numbers, is of the sample mean to the population mean $\endgroup$
    – Henry
    Jun 3 '16 at 21:05

I did some research and it seems like s ample variance (is this even called sample variance?) should be divided $n−1$ and not $n$ as seen in the solution

You are confusing the (true) variance of a random variable with the estimator (or sample variance) of that variable. Forget about that for a moment.

The sample mean $\overline{X}_n=\frac{X_1 +X_2+X_n}{n}$ is (can be used as) an estimator of the true mean $\mu = E(X)$. Futher, notice that $\overline{X}_n$ is in itself a random variable (in informal terms: it gives different -"random"- values in different trials; same as $X$ is a random variable). Instead, $\mu = E(X)$ is not a random variable, it's a constant paramenter. (It's typical that the estimator of a parameter is a random variable).

Now, because $\overline{X}_n$ is a random variable, we can speak of its mean and variance (same as we speak of the mean or variance of $X$).

It should be easy (but non trivial) to see that $E(\overline{X}_n)=E(X)$. But the variance of $Var(\overline{X}_n)$ does not equal $Var(X)$, but they are related by $Var(\overline{X}_n)=Var(X)/n$ (again, this is easy but non trivial).

Then, you apply the Chebyshev's inequality, but not to the variable $X$, but to the variable $\overline{X}_n$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.