Statistical test to evaluate if sample standard deviation differs from population standard deviation?

Question

Beginner in stats. I have read about tests that allow you to estimate with a certain degree of confidence whether a sample mean is consistent with a population mean. But I'd like to know is there an analagous test for calculating a confidence interval concerning whether a sample standard deviation is consistent with a population standard deviation? Also, how should you estimate the population standard deviation if you have a sample standard deviation?

Many thanks!

Answer 1

It's easier to answer in terms of population and sample variances. If data $X_1, \dots, X_n$ are $N(\mu, \sigma^2),$ and $S^2$ is defined by $(n-1)S^2 = \sum_{n=1}^n (X_i = \bar X)^2,$ then $(n - 1)S^2/\sigma^2 \sim CHISQ(n-1),$ the chi-squared distribution with degrees of freedom $DF = n-1.$

This makes it possible to find a 95% confidence interval for $\sigma^2$ as $((n-1)S^2/U, (n-1)S^2/L),$ where $L$ and $U$ cut 2.5% from the lower and upper tail of $CHISQ(n-1),$ respectively. Take square roots of the endpoints to get a 95% confidence interval for the population standard deviation $\sigma.$

Similarly, this distributional relationship between $S^2$, $\sigma^2$, and $CHISQ(n-1)$ allows one- and two-sided tests of $H_0: \sigma = \sigma_0.$ You can find proofs and details in some elementary-level statistics books, and most intermediate-level ones. Online see the NIST handbook: http://www.itl.nist.gov/div898/handbook/eda/section3/eda358.htm.

The sample variance $S^2$ is an unbiased estimator of the population variance $\sigma^2.$ (To make this so is one reason for using $n-1$ in the denominator of $S^2.$) However, expectation is a linear operator and unbiasedness does not survive taking square roots, so that $E(S) < \sigma.$ However, the amount of bias is slight and tends to 0 with increasing sample size $n.$ For example, with normal data, $E(S) \approx 0.97\sigma$ for $n = 10.$ It is common practice to estimate $\sigma$ by $S.$

Addendum: For $n$ iid normal observations $E(S) = \sigma\sqrt{\frac{2}{n-1}}\Gamma(\frac{n}{2})/\Gamma(\frac{n-1}{2}).$