Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Normally, in AP Stats we were taught that to find the standard deviation of a new set of data, for example: to compute $stdv(A - B)$, you say

$$stdv(A - B) = \sqrt{\text{var}(A) + \text{var}(B)}$$

This normally works all fine and dandy. For this set of data I'm dealing with for our final project, though, the calculator says the standard deviation of $84.1603$. The standard deviation of $A$ is $66.2304$, and of $B$ is $54.0266$. When I compute the standard deviation, I cannot get anything other than $85.4713$. Am I doing something wrong? This is crucial for the project, as this miscalculation makes the difference between this portion of our experiment being statistically significant or not. Please help.

share|improve this question
add comment

1 Answer 1

The variance of the probability distribution of a random variable $X$ is $$ \mathbb E((X-\mu)^2)\text{ where } \mu=\mathbb E X. $$ If the random variable takes values $x_1,\ldots,x_n$ with equal probability, then the variance is $$ \frac1n\sum_{i=1}^n (x_i-\bar x)^2\text{ where }\bar x=\frac{x_1+\cdots+x_n}{n}. $$

But many software packages compute instead $$ \frac{1}{n-1}\sum_{i=1}^n (x_i-\bar x)^2\text{ where }\bar x=\frac{x_1+\cdots+x_n}{n}. $$

This makes sense at most only if $x_1,\ldots,x_n$ is a random sample from a large population and one is using this quantity to estimate the unobserved variance of the whole population. Often one sees this expression with $n-1$ instead of $n$ called the "sample variance" and the version with $n$ called the "population variance". The latter terminology makes sense only when $n$ is the size of the whole population. The additivity of variances simply doesn't work when $n-1$ is used instead of $n$. The replacement of $n$ with $n-1$ is called Bessel's correction. It's purpose is to make the estimate unbiased. Google the terms "Bessel's correction" and "unbiased estimator".

Making estimators unbiased is of less value than the terminology makes it sound. The value of the constant $c$ that makes the mean squared error of estimation of $$ c\sum_{i=1}^n (x_i-\bar x)^2 $$ as small as possible is $c=1/(n+1)$. Notice "$+$" rather than "$-$".

If you can tell me what sort of statistical test you're doing I might be able to say more.

share|improve this answer
    
Thanks a ton! Your response makes a lot of sense. The n-1 could make a difference in this situation. This has saved my project from complete failure. Thanks! –  Oracular Jun 14 '13 at 0:53
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.