Why do we approximate sample stdev using sample stdev / sqrt(n) when population stdev is unknown? In this stackoverflow stdev estimation question, we are discussing about when to use z-distribution and t-distribution. Now, I'm getting confused here why are we taking the indirect & apparently longer path for finding sample z-score?
For a given set of sample and population, $sample \ z_{score}= \frac{x - \mu_{sample}}{\sigma_{sample}}$.
Now, we go around to estimate $\sigma_{sample} = \frac{\sigma_{population}}{\sqrt{n}} \ \ \ \ \ \ \  ... [1]\ $ where $n$ = sample size.
Now, my question is following: If we want $\sigma_{sample}$, why not simply find it using formula:
$$\sigma = \frac{\sum_{i=1}^{n}(x_{i}-\mu)^2}{N-1}$$
Also, this will be having a different answer from our estimation [1]'s $\sqrt{n}$ factor.
Please help me if I'm missing something major over here.
 A: I think you are confusing the standard deviation of the population and the standard deviation of the sample average.
Let's suppose $X_1,...,X_n$ is a random sample (i.e. they are IID) of heights of people from a population, where their mean is $E[X_i]=\mu$ and variance is $\text{Var}(X_i)=\sigma^2$. Note that $\sigma^2$ intuitively captures uncertainty associated with a single draw from our population. Generally, we won't know $\sigma^2$, but an unbiased estimator for $\sigma^2$ that we can compute from our data is
$$S^2\equiv \frac{1}{n-1}\sum_{i=1}^n (X_i-\bar X_n)^2 $$
where $\bar X_n\equiv \frac{1}{n} \sum_{i=1}^n X_i$ is the sample average.
Now, the sample average is itself a random variable, and you can work out its variance is given by
$$\text{Var}(\bar X_n)=\frac{\sigma^2}{n},$$
which can be estimated by $S^2/n.$ Note $\sigma^2/n$ intuitively captures uncertainty associated with the sample average, which decreases as we use a larger sample.
For the purposes of constructing a z or t statistic, we care about the sample average, because the central limit theorem, roughly, is a statement about approximate normality of the sample average. And by the above discussion, the appropriate object to standardize $\bar X_n$ by would be $\sigma/\sqrt n$ (z test) or $S/\sqrt n$ (t test).
