Given $X_1,...,X_n$ iid to a certain distribution (not necessarily normal), with $\mathbb{E}(X_i)=\mu$ and $\mathbb{V}(X_i)=\sigma^2$, I'm trying to deduce the standard and mean squared error of the estimator $\widehat{\sigma}^2=S_n^2$, where $S_n^2$ is the sample variance, given by$$ S_n^2=\frac{1}{n-1}\sum_{i=1}^n(X_i-\overline{X}_n)^2. $$ To do so I need its variance, $\mathbb{V}(S_n^2)$. Since I know the expectation $\mathbb{E}(S_n^2)=\sigma^2$, I started by expanding$$ \mathbb{V}(S_n^2)=\mathbb{E}(S_n^4)-(\mathbb{E}(S_n^2))^2=\mathbb{E}(S_n^4)-\sigma^4 $$ but I'm stuck with the expansion of the term $\mathbb{E}(S_n^4)$. Any ideas?

P.S. - I saw user940's answer on Variance of sample variance?, but I was looking for a different approach, also not assuming normal distributed random variables.

  • $\begingroup$ Emm how different approach you want? I have typed a 2-3 pages of full derivation for this starting from the Casella Berger Ex hints also. So basically it was just an expansion. If you start from this definition of $S^2$, I believe you will end up facing with the same expansion problem, but slightly longer. $\endgroup$ – BGM Oct 17 '17 at 11:02
  • $\begingroup$ You're right, I guess there is no way around it. I ended up doing the expansion and left my solution below. $\endgroup$ – sam wolfe Oct 17 '17 at 12:00

In the meantime I think I solved it. Here's my solution:

Let $\mu_k$ denote the $k$th central momentum of $X_i$, i.e, $\mu_k=\mathbb{E}((X_i-\mu)^k)$, and $Z_i\equiv X_i-\mu$ for all $i$. Thus $\mathbb{E}(Z_i)=0$. Since$$ \mathbb{V}(S_n^2)=\mathbb{E}(S_n^4)-(\mathbb{E}(S_n^2))^2=\mathbb{E}(S_n^4)-\sigma^4, $$we derive an expression of $\mathbb{E}(S_n^4)$ in terms of $n$ and the moments. We can rewrite $S_n^2$ as\begin{align*} S_n^2=\frac{n\sum_{i=1}^n Z_i^2-(\sum_{i=1}^nZ_i)^2}{n(n-1)}. \end{align*} Squaring leads to\begin{align*} S_n^4=\frac{n^2(\sum_{i=1}^nZ_i^2)^2-2n(\sum_{i=1}^nZ_i^2)(\sum_{i=1}^nZ_i)^2+(\sum_{i=1}^nZ_i)^4}{n^2(n-1)^2} \end{align*} and so\begin{align*} \mathbb{E}(S_n^4)=\frac{n^2\mathbb{E}((\sum_{i=1}^nZ_i^2)^2)-2n\mathbb{E}\left((\sum_{i=1}^nZ_i^2)(\sum_{i=1}^nZ_i)^2 \right)+\mathbb{E}((\sum_{i=1}^n Z_i)^4)}{n^2(n-1)^2}. \end{align*} Since $Z_1,...,Z_n$ are independent, we have that, for distinct $i,j,k$,\begin{align*} \mathbb{E}(Z_iZ_j)=0,\hspace{5mm}\mathbb{E}(Z_i^3Z_j)=0,\hspace{5mm}\mathbb{E}(Z_i^2Z_jZ_k)=0 \end{align*} and\begin{align*} E(Z_i^2Z_j^2)=\mu_2^2=\sigma^4,\hspace{5mm}\mathbb{E}(Z_i^4)=\mu_4. \end{align*} Then, with simple algebraic simplifications, it can be shown that the following holds\begin{align*} \mathbb{E}((\sum_{i=1}^nZ_i^2)^2)&=n\mu_4+n(n-1)\sigma^4,\\ \mathbb{E}\left((\sum_{i=1}^nZ_i^2)(\sum_{i=1}^nZ_i)^2 \right)&=n\mu_4+n(n-1)\sigma^4,\\ \mathbb{E}((\sum_{i=1}^n Z_i)^4)&=n\mu_4+3n(n-1)\sigma^4. \end{align*} Substituting these into the expansion of $\mathbb{E}(S_n^4)$ and simplifying leads to\begin{align*} \mathbb{E}(S_n^4)=\frac{(n-1)\mu_4+(n^2-2n+3)\sigma^4}{n(n-1)} \end{align*} and so\begin{align}\label{var} \mathbb{V}(S_n^2)=\mathbb{E}(S_n^4)-\sigma^4=\frac{(n-1)\mu_4+(n^2-2n+3)\sigma^4}{n(n-1)}-\sigma^4=\frac{\mu_4}{n}-\frac{\sigma^4(n-3)}{n(n-1)}. \end{align}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.