Understanding properties of the standard deviation of a statistic

Question

In my textbook I have read:

Let $Y_1, Y_2, \dots, Y_n$ be a random sample of size $n$ from a normal distribution with mean $\mu$ and variance $\sigma^2$. Then $$ \bar{Y} = \frac{1}{n} \sum_{i=1}^n{Y_i}$$ is normally distributed with mean $\mu_{\bar{Y}} = \mu$ and variance $\sigma_{\bar{Y}}^2= \sigma^2/n$.

A proof of this claim is provided right after and I understand the proof.

However there's something I seem to be not understanding.

From this theorem I conclude that as $n$ tends to infinity, the standard deviation tends to 0.

But my expecation is, if we sample 100% of the elements of a population, the standard deviation fo the sample should be equal to the standard deviation of the population.

Where am I missunderstanding this theorem? Because it simply doesn't make sense that the standard deviation goes to 0 as the sample size increases. it should tend towards the standard deviation of the population.

Answer 1

Maybe it helps to do this for $n = 2$, as a start. There are three key results that will help.

(1) If $X_1$ and $X_2$ have $E(X_1) = \mu_1$ and $E(X_2) = \mu_2,$ then $$E(a_1X_1 = a_2X_2) = a_1E(X_1) + a_2E(X_2) = a_1\mu_1 + a_2\mu_2.$$ For $\bar X = \frac 1 2 (X_1 + X_2),$ take $a_1 = a_2 = 1/2$ and $\mu_1 = \mu_2 = \mu,$ to get $E(\bar X) = \mu.$ So the mean is right.

(2) Similarly, if $X_1$ and $X_2$ are independent and have $Var(X_1) = \sigma_1^2$ and $Var(X_2) = \sigma_2^2,$ then $$Var(a_1X_1 = a_2X_2) = a_1^2 Var(X_1) + a_2^2 Var(X_2) = a_1^2\sigma_1^2 + a_2^2\sigma_2^2.$$ For $\bar X = \frac 1 2 (X_1 + X_2),$ take $a_1 = a_2 = 1/2$ and $\sigma_1^2 = \sigma_2^2 = \sigma^2,$ to get $Var(\bar X) = \sigma^2/2.$ So the variance is right.

(3) If $X_1 \sim \mathsf{Norm}(\mu_1, \sigma_1)$ and independently $X_2 \sim \mathsf{Norm}(\mu_2, \sigma_2),$ then you can use moment generating functions to prove that $X_1 + X_2 \sim \mathsf{Norm}\left(\mu_1 + \mu_2,\, \sqrt{\sigma_1^2 + \sigma_2^2}\right).$ So the normal distribution part is right. That is, the sum of two independent normal random variables is a normal random variable; the means add and the variances add (not the standard deviations).

Finally, you have $\bar X = \frac 1 2(X_1 + X_2) \sim \mathsf{Norm}(\mu, \sigma^2/2).$

So there is lot involved in the statement you posted. Without some careful explanation before or after this statement pointing out connections to the three results I have mentioned, this is a pretty big chunk to absorb at first glance. But, taken separately, each part of the explanation is simple enough.

I hope you can find statements in your text similar to my (1), (2), and (3) and make sure you see how they apply. Maybe some proofs are postponed until later, but make what linkages you can for now, and revisit the statement later as necessary.

---

Your second question seems to be about estimating the population parameters $\mu$ and $\sigma$ from a large sample. For large $n,$ you have $\bar X_n$ very close to $\mu.$ So you want $\bar X_n$ to have a small variance in order to make $\bar X_n$ have good precision as an estimate of $\mu.$

Estimating the population variance $\sigma^2$ by the sample variance $S_n^2$ of a large sample is a separate issue. First, one can show thate $E(S_n^2) = \sigma^2.$ Second, one can show that $\frac{(n-1)S_n^2}{\sigma^2} \sim \mathsf{Chisq}(df = n-1).$ Without going into details prematurely, this implies that $S_n^2$ gets ever closer to $\sigma^2$ as $n$ increases. I sampled $n=1000$ observations from $\mathsf{Norm}(\mu = 100, \sigma = 10)$ and got $S_{1000}^2 = 100.2097.$ Using the chi-squared distribution this gives $(91.97, 109.61)$ as a 95% confidence interval for $\sigma^2$ and [taking square roots] also $(9.59, 10.47)$ as a 95% CI for $\sigma.$

.

Answer 2

"From this theorem I conclude that as n tends to infinity, the standard deviation tends to $0$".

It's the variance in the sample means that is decreasing not the standard deviation of the samples.