Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

What is the variance of the sample variance? In other words I am looking for $\mathrm{Var}(S^2)$.

I have started by expanding out $\mathrm{Var}(S^2)$ into $E(S^4) - [E(S^2)]^2$

I know that $[E(S^2)]^2$ is $\sigma$ to the power of 4. And that is as far as I got.

share|improve this question
    
Your expressions are very difficult to read. You need to edit and present your question in a better way. –  smanoos Oct 16 '11 at 4:44
1  
One way of expressing $Var(S^2)$ is given on the Wikipedia page for variance. –  Mike Spivey Oct 16 '11 at 4:50
    
It doesn't show how they derived it. –  MathMan Oct 16 '11 at 4:51
    
The solution to the question is in many books. You can easily find it. –  smanoos Oct 16 '11 at 5:17
    
There is a derivation on MathWorld's Sample Variance Distribution page. They use the "divide by $N$" convention rather than the "divide by $N-1$" convention, though, so you might have to adjust for that. –  Mike Spivey Oct 16 '11 at 5:39

4 Answers 4

up vote 12 down vote accepted

Maybe, this will help. Let's suppose the samples are taking from a normal distribution. Then using the fact that $\frac{(n-1)S^2}{\sigma^2}$ is a chi squared random variable with $(n-1)$ degrees of freedom, we get $$\begin{align*} \text{Var}~\frac{(n-1)S^2}{\sigma^2} & = \text{Var}~\chi^{2}_{n-1} \\ \frac{(n-1)^2}{\sigma^4}\text{Var}~S^2 & = 2(n-1) \\ \text{Var}~S^2 & = \frac{2(n-1)\sigma^4}{(n-1)^2}\\ & = \frac{2\sigma^4}{(n-1)}, \end{align*}$$

where we have used that fact that $\text{Var}~\chi^{2}_{n-1}=2(n-1)$.

Hope this helps.

share|improve this answer
3  
Remember that $(n-1)S^2/\sigma^2$ is only guaranteed to be $\chi^2$ when the sample is taken from a normal distribution, though. –  Mike Spivey Oct 16 '11 at 6:09
    
Thanks Mike. I've edited my answer to reflect what you said in your comment. –  Nana Oct 16 '11 at 6:15
    
The question posed is a general one, whereas the answer is distribution-specific. Not appropriate, I am afraid. –  wolfies Apr 26 '13 at 7:49
    
@afsdfdfsaf Perhaps, you should ask that as a separate question. –  Nana Apr 12 at 18:48

The best way to understand what the variance of a sample looks like is to derive it from scratch.

On the following site you will find the complete derivation (it goes over 70 steps) of the sample variance. It takes a bit of time to fully understand how it is working, but if one goes over the whole derivation several times it becomes quite clear.

You will also understand better, why the proposed sample variance estimator is unbiased.

http://economictheoryblog.wordpress.com/2012/06/28/latexlatexs2/

share|improve this answer

There can be some confusion in defining the sample variance ... 1/n vs 1/(n-1). The OP here is, I take it, using the sample variance with 1/(n-1) ... namely the unbiased estimator of the population variance, otherwise known as the second h-statistic:

h2 = HStatistic[2][[2]] 

These sorts of problems can now be solved by computer. Here is the solution using the mathStatica add-on to Mathematica. In particular, we seek the Var[h2], where the variance is just the 2nd central moment, and express the answer in terms of central moments of the population:

CentralMomentToCentral[2, h2] 

We could just as easily find, say, the 4th central moment of the sample variance, as:

CentralMomentToCentral[4, h2]

share|improve this answer

Here's a general derivation that does not assume normality.

Let's rewrite the sample variance $S^2$ as an average over all pairs of indices: $$S^2={1\over{n\choose 2}}\sum_{\{i,j\}} {1\over2}(X_i-X_j)^2.$$ Since $\mathbb{E}[(X_i-X_j)^2/2]=\sigma^2$, we see that $S^2$ is an unbiased estimator for $\sigma^2$.

The variance of $S^2$ is the expected value of $$\left({1\over{n\choose 2}}\sum_{\{i,j\}} \left[{1\over2}(X_i-X_j)^2-\sigma^2\right]\right)^2.$$

When you expand the outer square, there are 3 types of cross product terms $$\left[{1\over2}(X_i-X_j)^2-\sigma^2\right] \left[{1\over2}(X_k-X_\ell)^2-\sigma^2\right]$$ depending on the size of the intersection $\{i,j\}\cap\{k,\ell\}$.

  1. When this intersection is empty, the factors are independent and the expected cross product is zero.

  2. There are $n(n-1)(n-2)$ terms where $|\{i,j\}\cap\{k,\ell\}|=1$ and each has an expected cross product of $(\mu_4-\sigma^4)/4$.

  3. There are ${n\choose 2}$ terms where $|\{i,j\}\cap\{k,\ell\}|=2$ and each has an expected cross product of $(\mu_4+\sigma^4)/2$.

Putting it all together shows that $$\mbox{Var}(S^2)={\mu_4\over n}-{\sigma^4\,(n-3)\over n\,(n-1)}.$$ Here $\mu_4=\mathbb{E}[(X-\mu)^4]$ is the fourth central moment of $X$.

share|improve this answer
1  
a related question on stats.SE asks provides a different solution, and asks for a reference, your input would be appreciated: stats.stackexchange.com/q/29905/2750 –  Abe Jun 6 '12 at 16:56
    
@Abe Sorry, I don't have any references or worthwhile input. The above is a solution that I made up to teach my students. –  Byron Schmuland Jun 6 '12 at 19:07
1  
thanks, an answer to the stats.SE question solved my confusion: the discrepancy was use of kurtosis ($\mu_4$, the fourth central moment) vs excess kurtosis ($\kappa = \frac{\mu_4}{\sigma^4} -3$); one reference is Mood Graybill and Boes, 1974, Introduction to the Theory of Statistics –  Abe Jun 6 '12 at 22:39

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.