# Understanding properties of the standard deviation of a statistic

In my textbook I have read:

Let $Y_1, Y_2, \dots, Y_n$ be a random sample of size $n$ from a normal distribution with mean $\mu$ and variance $\sigma^2$. Then $$\bar{Y} = \frac{1}{n} \sum_{i=1}^n{Y_i}$$ is normally distributed with mean $\mu_{\bar{Y}} = \mu$ and variance $\sigma_{\bar{Y}}^2= \sigma^2/n$.

A proof of this claim is provided right after and I understand the proof.

However there's something I seem to be not understanding.

From this theorem I conclude that as $n$ tends to infinity, the standard deviation tends to 0.

But my expecation is, if we sample 100% of the elements of a population, the standard deviation fo the sample should be equal to the standard deviation of the population.

Where am I missunderstanding this theorem? Because it simply doesn't make sense that the standard deviation goes to 0 as the sample size increases. it should tend towards the standard deviation of the population.

• This statement will imply the weak law of large numbers for the specific collection of random variables $Y_1,Y_2,\dots$ as described in the problem. "If we sample 100% of the elements of a population, the standard deviation for the sample should be equal to the standard deviation of the population." You seem to be ignoring the division by $n$ which occurs in the definition of $\overline{Y}$. – JMoravitz Jun 24 '18 at 21:33
• For a different, slightly unrelated example but more tangible perhaps, suppose each of $Y_1,Y_2,\dots$ were Bernouli random variables (i.e. only take value $0$ or $1$) representing coin flips: 0 if tails, 1 if heads. Yes, if we look at the variance of the total number of heads, it will become a large number... but we are instead looking at the variance in terms of a ratio compared to the total number of flips. – JMoravitz Jun 24 '18 at 21:40

Maybe it helps to do this for $n = 2$, as a start. There are three key results that will help.

(1) If $X_1$ and $X_2$ have $E(X_1) = \mu_1$ and $E(X_2) = \mu_2,$ then $$E(a_1X_1 = a_2X_2) = a_1E(X_1) + a_2E(X_2) = a_1\mu_1 + a_2\mu_2.$$ For $\bar X = \frac 1 2 (X_1 + X_2),$ take $a_1 = a_2 = 1/2$ and $\mu_1 = \mu_2 = \mu,$ to get $E(\bar X) = \mu.$ So the mean is right.

(2) Similarly, if $X_1$ and $X_2$ are independent and have $Var(X_1) = \sigma_1^2$ and $Var(X_2) = \sigma_2^2,$ then $$Var(a_1X_1 = a_2X_2) = a_1^2 Var(X_1) + a_2^2 Var(X_2) = a_1^2\sigma_1^2 + a_2^2\sigma_2^2.$$ For $\bar X = \frac 1 2 (X_1 + X_2),$ take $a_1 = a_2 = 1/2$ and $\sigma_1^2 = \sigma_2^2 = \sigma^2,$ to get $Var(\bar X) = \sigma^2/2.$ So the variance is right.

(3) If $X_1 \sim \mathsf{Norm}(\mu_1, \sigma_1)$ and independently $X_2 \sim \mathsf{Norm}(\mu_2, \sigma_2),$ then you can use moment generating functions to prove that $X_1 + X_2 \sim \mathsf{Norm}\left(\mu_1 + \mu_2,\, \sqrt{\sigma_1^2 + \sigma_2^2}\right).$ So the normal distribution part is right. That is, the sum of two independent normal random variables is a normal random variable; the means add and the variances add (not the standard deviations).

Finally, you have $\bar X = \frac 1 2(X_1 + X_2) \sim \mathsf{Norm}(\mu, \sigma^2/2).$

So there is lot involved in the statement you posted. Without some careful explanation before or after this statement pointing out connections to the three results I have mentioned, this is a pretty big chunk to absorb at first glance. But, taken separately, each part of the explanation is simple enough.

I hope you can find statements in your text similar to my (1), (2), and (3) and make sure you see how they apply. Maybe some proofs are postponed until later, but make what linkages you can for now, and revisit the statement later as necessary.

Your second question seems to be about estimating the population parameters $\mu$ and $\sigma$ from a large sample. For large $n,$ you have $\bar X_n$ very close to $\mu.$ So you want $\bar X_n$ to have a small variance in order to make $\bar X_n$ have good precision as an estimate of $\mu.$

Estimating the population variance $\sigma^2$ by the sample variance $S_n^2$ of a large sample is a separate issue. First, one can show thate $E(S_n^2) = \sigma^2.$ Second, one can show that $\frac{(n-1)S_n^2}{\sigma^2} \sim \mathsf{Chisq}(df = n-1).$ Without going into details prematurely, this implies that $S_n^2$ gets ever closer to $\sigma^2$ as $n$ increases. I sampled $n=1000$ observations from $\mathsf{Norm}(\mu = 100, \sigma = 10)$ and got $S_{1000}^2 = 100.2097.$ Using the chi-squared distribution this gives $(91.97, 109.61)$ as a 95% confidence interval for $\sigma^2$ and [taking square roots] also $(9.59, 10.47)$ as a 95% CI for $\sigma.$

.

• This is not the issue. You have provided the same proof as my textbook. I understand the arithmetic. My problem lies in some missconception about the variables in the theorem and the role they play. Because we both KNOW that if you fully sample the space the standard deviation is just $\sigma$. In toher words I am not confused about the math, I am not reading the english properly somewhere. – Makogan Jun 24 '18 at 22:29
• Just added a second part to my answer. Maybe that will help. // This math is really based on a hypothetical infinite population, so you would never sample 100% of the population, but you can get very good estimates by taking suitably large samples. – BruceET Jun 24 '18 at 22:48
• I think I am not being able to express my query properly. I have given an upvote for efforth but nothing in your reply answers my question. I know my claim that a huge sample from a population should have a SD similar to the theoreticla one is true. This theorem must also be true. It follows that my understanding of what the theorem is talking about is ppoor. In other words, my problem is not with the amth itself. Is about understanding the role of the different variables in the theorem. – Makogan Jun 24 '18 at 22:56
• It is good that you are thinking ahead about the implications of the theory and about theorems to be proved later in your course. The trouble with thinking ahead is that you may not yet have the vocabulary to ask for just the clarification you want. Presumably that will come. – BruceET Jun 24 '18 at 23:01
• I am not thinking ahead at all. Like when I read your reply I understand the arithmetic perfectly. My issue is not understanding the math. My issue is relating the terms in english with what the theorem is stating. Like somwhere I am mixing things up and this is what is causing my confusion. The problme is, if I kew where my miss understanding lies, I wouldn't need to ask the question to ebgin with :p – Makogan Jun 24 '18 at 23:16

"From this theorem I conclude that as n tends to infinity, the standard deviation tends to $0$".

It's the variance in the sample means that is decreasing not the standard deviation of the samples.