Why does the central limit theorem imply that the standard deviation approaches $\frac{\sigma}{\sqrt{n}}$?

According to the central limit theorem, if one takes random samples of size $n$ from a population of mean $\mu$ and standard deviation $\sigma$, then as $X$ gets large, $X$ approaches the normal distribution with mean $\mu$ and standard deviation $\frac{\sigma}{\sqrt{n}}$.

$\frac{\sigma}{\sqrt{n}}$ doesn't make sense to me. Lets look at the extreme case. Say my sample consists of the entire population. Then, shouldn't my standard deviation be just $\sigma$ instead of $\sigma/\text{(population size)}$?

• If we are sampling independently (so with finite populations, with replacement) then the variance of the sample mean is $\sigma^2/n$. Nothing to do with near normality. In the case of finite population, and sampling without replacement, the variance is $\le \sigma^2/n$. Oct 4 '13 at 22:08

If $X$ represents here the sample mean $\bar X_n$, then the Central Limit Theorem says that the quantity
$$Z = \sqrt n(\bar X_n-\mu)$$ tends in distribution to $N(0,\sigma^2)$ as $n$ tends to infinity, and then by abusing notation and asymptotics, we write
$$\bar X_n = \frac{1}{\sqrt n}Z + \mu$$ which gives us that $\bar X_n \approx N(\mu,( \frac{\sigma}{\sqrt n})^2)$.
...which in a sense holds for some "intermediate range" of $n$ - because if $n$ truly passes over to infinity, then the distribution collapses to a single point, since the variance goes to zero (which is as it should).
Your $X$ is more commonly denoted ${\bar X}$, because it is the average of all observations. And intuitively, it seems likely that the average of many observations is closer to the real average $\mu$ of the population than the average of few observations. That is (part of) what the central limit theorem says: that you're more sure to be near the real average when you take more samples.