# Intuitive Explanation of Bessel's Correction

When calculating a sample variance a factor of $$N-1$$ appears instead of $$N$$ (see this link ). Does anybody have an intuitive way of explaining this to students who need to use this fact but maybe haven't taken a statistics course?

• This is essentially a duplicate of Sample Standard Deviation vs. Population Standard Deviation. Sep 1, 2011 at 19:04
• But the answer there just says that there is a bias and here's how you correct it. Is there a way to explain the correction intuitively (yes that's a little vague). Sep 1, 2011 at 19:07
• There is an attempt at an intuitive explanation in the second part of that answer. Does that help? Sep 1, 2011 at 19:08
• The concrete example I cite in my answer below can be understood without even using algebra. Sep 1, 2011 at 20:02
• The fact that correction for bias can sometimes be a very bad thing to do seems not widely understood among non-statisticians, including some who teach statistics. In this example of estimating variance, by the usual mean-squared error criterion, and assuming the sample is from a normally distributed population, the unbiased estimator is only slightly worse than the biased estimator, so it's not a good example to illustrate that point. But the idea that it is the unbiased estimator that is worse may fail to be as widely appreciated as it could be. Sep 1, 2011 at 20:11

http://en.wikipedia.org/wiki/Bessel%27s_correction

The Wikipedia article linked to above has a section (written by me) titled "The source of the bias". It explains it via a concrete example.

But note also that correcting for bias, when it can be done, is not always a good idea. I wrote this paper about that: http://arxiv.org/pdf/math/0206006.pdf

• Thank you. (....extra line because the software does not allow short comments....) Sep 1, 2011 at 21:21
• I use dollars  Sep 1, 2011 at 21:28
• (to circumvent the minimum character requirement) And while I haven't seen anyone else use this, I'm not opposed to sharing it with such a generous (mathematical) contributor! Sep 1, 2011 at 21:29
• @TheChaz: Unicode zero width spaces work much better, and don't have the formatting issues like in your comment above. Sep 16, 2013 at 14:27
• This is great explanation for someone like me who is not a statics person but need an intuition. Thanks a lot. Nov 23, 2013 at 17:35

To me, the main idea is that the sample mean is not the distribution (or population) mean. The sample mean is "closer" to the sample data than the distribution mean, so the variance computed is smaller.

Suppose the distribution mean is $m_d$. The sum of $n$ variates (the sample) is $n m_s$, where $m_s$ is the sample mean. Recall that the mean and variance of a sum of variates are the sum of the means and sum of the variances of the variates. That is, the distribution mean of the sum of $n$ variates is $n m_d$ and the distribution variance of the sum of $n$ variates is $n v_d$. In other words, $$\mathrm{E}[(n m_s-n m_d)^2]=n v_d$$ or equivalently, $$\mathrm{E}[(m_s-m_d)^2]=\frac{1}{n}v_d$$ Let us compute the expected sample variance as \begin{align} &\mathrm{E}[v_s]\\ &=\mathrm{E}\left[\frac{1}{n}\sum_{k=1}^n(x_k-m_s)^2\right]\\ &=\mathrm{E}\left[\frac{1}{n}\sum_{k=1}^n\left((x_k-m_d)^2+2(x_k-m_d)(m_d-m_s)+(m_d-m_s)^2\right)\right]\\ &=\mathrm{E}\left[\frac{1}{n}\sum_{k=1}^n\left((x_k-m_d)^2+2(m_s-m_d)(m_d-m_s)+(m_d-m_s)^2\right)\right]\\ &=\mathrm{E}\left[\frac{1}{n}\sum_{k=1}^n\left((x_k-m_d)^2-(m_d-m_s)^2\right)\right]\\ &=v_d-\frac{1}{n}v_d\\ &=\frac{n{-}1}{n}v_d \end{align} Thus, $$v_d=\frac{n}{n{-}1}\mathrm{E}[v_s]$$ This is why, to estimate the distribution variance, we multiply the sample variance by $\frac{n}{n{-}1}$. Thus, it appears as if we are dividing by $n{-}1$ instead of $n$.

• +1, though for an intuitive explanation, you could have stopped after the first paragraph. Sep 1, 2011 at 22:51
• @Henry: yes, I know. Since the derivation follows the idea of my intuitive explanation, I thought I would attach it for anyone who is interested.
– robjohn
Sep 2, 2011 at 3:27
• sorry to bump, but how did you justify replacing $x_k-m_d$ with $m_s-m_d$? I mean it makes sense their expectations are equal, but I was a little confused. Dec 27, 2014 at 20:45
• @oldrinb: I think that you are asking about the equation $$\mathrm{E}\left[\frac1n\sum_{k=1}^nx_k\right] =\mathrm{E}\left[\frac1n\sum_{k=1}^nm_s\right]$$ That follows because $$\frac1n\sum_{k=1}^nm_s =m_s =\frac1n\sum_{k=1}^nx_k$$
– robjohn
Dec 28, 2014 at 14:01
• You use $m_s$ for what is commonly denoted by $\bar{X}$ (the sample mean), right? It's defined as a sum of $n$ random variables $X_i$ divided by $n$. Jan 9, 2016 at 22:25

The question refers to "explaining this to students who need to use this fact but maybe haven't taken a statistics course". If they're more advanced than those who will understand the example that I mentioned that does not require algebra beyond expanding $(a+b)^2$, maybe a couple of other points of view are worth looking at.

We can write $$\begin{bmatrix}x_1 \\ \vdots \\ x_n\end{bmatrix} = \begin{bmatrix}\overline{x} \\ \vdots \\ \overline{x} \end{bmatrix} + \begin{bmatrix} x_1 - \overline{x} \\ \vdots \\ x_n - \overline{x}\end{bmatrix},$$ and notice that the two vectors being added are the orthogonal projections of the sum onto spaces of dimensions $1$ and $n-1$. The expected value of the first summand is $\mu$ times a column of $1$s, and the expected value of the second summand is $0$. So rotate the coordinate system so that this becomes $$\begin{bmatrix}u_1 \\ \vdots \\ u_n\end{bmatrix} = \begin{bmatrix} u_1 \\ 0 \\ 0 \\ \vdots \\ 0 \end{bmatrix} + \begin{bmatrix} 0 \\ u_2 \\ u_3 \\ \vdots \\ u_n \end{bmatrix}.$$ The expected value of the first entry in the first summand is $\mu\sqrt{n}$. The expected value of every entry in the second summand is $0$. The expected value of the square of the norm of the second vector is $n-1$ times the expected value of the square of any of its entries. That's where the $n-1$ comes from. Notice that $$\underbrace{\sum_{i=1}^n (x_i - \overline{x})^2}_{n\text{ terms}} = \underbrace{\sum_{i=2}^n u_i^2}_{n-1\text{ terms}}.$$

If students do know some probability theory, the above can also explain why $\sum_{i=1}^n (X_i - \overline{X})^2/\sigma^2$ has a chi-square distribution with $n-1$ degrees of freedom when there are certain assumptions about normal distribution and about independence. (I use capital $X$ this time since it's a random variable.) It can also explain why $\overline{X}$ is actually independent of that chi-square random variable.

Another thing that is sometimes useful in thinking about this topic is the algebraic identity $$\sum_{i=1}^n (x_i - \mu)^2 = n(\overline{x} - \mu)^2 + \sum_{i=1}^n (x_i - \overline{x})^2 \text{ where } \overline{x} = \frac{x_1+\cdots+x_n}n.$$ Clearly this implies that $$\sum_{i=1}^n (x_i - \mu)^2 \ge \sum_{i=1}^n (x_i - \overline{x})^2$$ with equality if and only if $\overline{x}=\mu$. This is of course the same thing as what was used in the concrete example in the Wikipedia article linked in my earlier answer, but stated in a way that will be understood by students who know more algebra than the expansion of $(a+b)^2$.

• Very nice explanation, the LA approach gives a different perspective. Jun 24, 2016 at 1:38
• @jeremyradcliff : Thank you. $\qquad$ Jun 24, 2016 at 16:40

The sum $\sum (x_i - a)^2$ is minimized when $a$ is the average of the $x_i$'s. The proof is a simple exercise in algebra (write sum as quadratic in $a$) or calculus (differentiate to get $\sum (x_i - a) = 0$).

Therefore, $\sum (x_i - \overline{x})^2 \leq \sum (x_i - \mu)^2$. The same inequality holds with averages in place of sums.

The average of the squared $(x_i - \mu)$'s is an unbiased estimate of the variance, since each term has that variance as its expected value. Replacing $\mu$ by $\overline{x}$ produces a smaller estimate of the variance, and the expected value of that estimate is lower than the expected value of the unbiased estimator with $\mu$. The latter is the variance, so the expression with $1/N$ will (on average) underestimate the variance.

To summarize: the use of $\overline{x}$ as a surrogate for $\mu$ causes a downward bias in estimating variance by $\sum (x_i - \overline{x})^2 / N$.

The main reason is that (unknown to many people!) Bessel's correction only gives an unbiased estimator for the variance of samples with replacement.

The result of this sampling with replacement is that you get "more of the same" numbers into the sample than are in the original population. Therefore your variance in the sample is lower on average compared to the original population.

This bias is corrected by dividing by a smaller number ($n-1$ instead of $n$) so that the variance gets bigger, i.e. unbiased again.

I just finished a purely visual explanation of the intuition of bessels at creatorguides.

Basically it happens because the sample mean can differ from the population mean, so the proof uses the binomial theorem to show this by creating the population covariance squares out of the sample covariance squares like the section in the wikipedia article.

The wikipedia article explains why it's n-1 and not n once you understand the gist of the need for it.

• Unfortuantely sciguides.com no longer exist and along with it the creatorguides. Nov 19, 2017 at 4:04

Here's a really simple example:

Say that you have a list of the heights of all the players in a sports league. Call that your "population" and calculate the variance. Now pick a sample of three players at random and calculate the variance of your sample. Chances are that your sample won't pick up the most extreme values, thereby causing the variance of the sample to be lower than the variance of the overall population.

In order to correct for that, you need to adjust your sample variance upward. That's why you use 1/(n-1) instead of 1/n. At small values of n (such as 3) the difference between these two will be pretty big. As your sample size increases, the difference gets smaller. This is because your sample is more likely to capture the extreme values in the population and therefore your sample variance will approach the population variance.

• This is a misleading explanation. If you pick a small sample, you expect to get a small spread, but you'll divide by the size of the sample in the end anyway, so that's not a reason for the resulting sample variance to be smaller. Indeed, your argument would still apply if you computed the variance with respect to the true population mean, but in this case the expected sample variance is equal to the population variance.
– user856
Nov 21, 2014 at 16:46