I have an HP 50g graphing calculator and I am using it to calculate the standard deviation of some data. In the statistics calculation there is a type which can have two values:

Sample Population

I didn't change it, but I kept getting the wrong results for the standard deviation. When I changed it to "Population" type, I started getting correct results!

Why is that? As far as I know, there is only one type of standard deviation which is to calculate the root-mean-square of the values!

Did I miss something?


There are, in fact, two different formulas for standard deviation here: The population standard deviation $\sigma$ and the sample standard deviation $s$.

If $x_1, x_2, \ldots, x_N$ denote all $N$ values from a population, then the (population) standard deviation is $$\sigma = \sqrt{\frac{1}{N} \sum_{i=1}^N (x_i - \mu)^2},$$ where $\mu$ is the mean of the population.

If $x_1, x_2, \ldots, x_N$ denote $N$ values from a sample, however, then the (sample) standard deviation is $$s = \sqrt{\frac{1}{N-1} \sum_{i=1}^N (x_i - \bar{x})^2},$$ where $\bar{x}$ is the mean of the sample.

The reason for the change in formula with the sample is this: When you're calculating $s$ you are normally using $s^2$ (the sample variance) to estimate $\sigma^2$ (the population variance). The problem, though, is that if you don't know $\sigma$ you generally don't know the population mean $\mu$, either, and so you have to use $\bar{x}$ in the place in the formula where you normally would use $\mu$. Doing so introduces a slight bias into the calculation: Since $\bar{x}$ is calculated from the sample, the values of $x_i$ are on average closer to $\bar{x}$ than they would be to $\mu$, and so the sum of squares $\sum_{i=1}^N (x_i - \bar{x})^2$ turns out to be smaller on average than $\sum_{i=1}^N (x_i - \mu)^2$. It just so happens that that bias can be corrected by dividing by $N-1$ instead of $N$. (Proving this is a standard exercise in an advanced undergraduate or beginning graduate course in statistical theory.) The technical term here is that $s^2$ (because of the division by $N-1$) is an unbiased estimator of $\sigma^2$.

Another way to think about it is that with a sample you have $N$ independent pieces of information. However, since $\bar{x}$ is the average of those $N$ pieces, if you know $x_1 - \bar{x}, x_2 - \bar{x}, \ldots, x_{N-1} - \bar{x}$, you can figure out what $x_N - \bar{x}$ is. So when you're squaring and adding up the residuals $x_i - \bar{x}$, there are really only $N-1$ independent pieces of information there. So in that sense perhaps dividing by $N-1$ rather than $N$ makes sense. The technical term here is that there are $N-1$ degrees of freedom in the residuals $x_i - \bar{x}$.

For more information, see Wikipedia's article on the sample standard deviation.

  • $\begingroup$ How does the formula $\frac{ \sigma}{\sqrt{n}}$ come from $s = \sqrt{\frac{1}{N-1} \sum_{i=1}^N (x_i - \bar{x})^2}$? $\endgroup$ – CodyBugstein Nov 13 '12 at 23:24
  • 4
    $\begingroup$ @Imray: It doesn't. They refer to two different things. The expression $\sigma/\sqrt{n}$ is the standard deviation of the mean $\bar{x}$ of the sample data. The expression $s = \sqrt{\frac{1}{N-1} \sum_{i=1}^N (x_i - \bar{x})^2}$ is the standard deviation of the sample data (so not the mean of the sample data). $\endgroup$ – Mike Spivey Nov 14 '12 at 3:57
  • 2
    $\begingroup$ "Since x¯ is calculated from the sample, the values of xi are on average closer to x¯ than they would be to μ" -- oh wow, thank you. I've read the explanation based on df a number of times, but this suddenly makes it intuitively clear in a different way. $\endgroup$ – octern Sep 6 '14 at 19:46
  • $\begingroup$ @MikeSpivey: Nice explanation. But if I know population mean μ then shouldn't I divide population variance by n-1 too? $\endgroup$ – Durin Sep 28 '14 at 20:09
  • 2
    $\begingroup$ @Durin: No. If you actually know $\mu$ and you want to calculate $\sigma$ with all $N$ finite values from the population, then you're not estimating anything. In this case $\sigma$ should be calculated using the straight-up average of the squared-deviations - dividing by $n$ instead of using the estimation-bias corrected version that divides by $n-1$. $\endgroup$ – Mike Spivey Sep 28 '14 at 20:47

protected by Zev Chonoles Jul 23 '16 at 4:13

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.