6
$\begingroup$

Given $X_1,...,X_n$ iid to a certain distribution (not necessarily normal), with $\mathbb{E}(X_i)=\mu$ and $\mathbb{V}(X_i)=\sigma^2$, I'm trying to deduce the standard and mean squared error of the estimator $\widehat{\sigma}^2=S_n^2$, where $S_n^2$ is the sample variance, given by$$ S_n^2=\frac{1}{n-1}\sum_{i=1}^n(X_i-\overline{X}_n)^2. $$ To do so I need its variance, $\mathbb{V}(S_n^2)$. Since I know the expectation $\mathbb{E}(S_n^2)=\sigma^2$, I started by expanding$$ \mathbb{V}(S_n^2)=\mathbb{E}(S_n^4)-(\mathbb{E}(S_n^2))^2=\mathbb{E}(S_n^4)-\sigma^4 $$ but I'm stuck with the expansion of the term $\mathbb{E}(S_n^4)$. Any ideas?

P.S. - I saw user940's answer on this question, but I was looking for a different approach, also not assuming normal distributed random variables.

$\endgroup$
2
  • $\begingroup$ Emm how different approach you want? I have typed a 2-3 pages of full derivation for this starting from the Casella Berger Ex hints also. So basically it was just an expansion. If you start from this definition of $S^2$, I believe you will end up facing with the same expansion problem, but slightly longer. $\endgroup$
    – BGM
    Oct 17, 2017 at 11:02
  • $\begingroup$ You're right, I guess there is no way around it. I ended up doing the expansion and left my solution below. $\endgroup$
    – sam wolfe
    Oct 17, 2017 at 12:00

1 Answer 1

13
$\begingroup$

In the meantime I think I solved it. Here's my solution:

Let $\mu_k$ denote the $k$th central momentum of $X_i$, i.e, $\mu_k=\mathbb{E}((X_i-\mu)^k)$, and $Z_i\equiv X_i-\mu$ for all $i$. Thus $\mathbb{E}(Z_i)=0$. Since$$ \mathbb{V}(S_n^2)=\mathbb{E}(S_n^4)-(\mathbb{E}(S_n^2))^2=\mathbb{E}(S_n^4)-\sigma^4, $$we derive an expression of $\mathbb{E}(S_n^4)$ in terms of $n$ and the moments. We can rewrite $S_n^2$ as\begin{align*} S_n^2=\frac{n\sum_{i=1}^n Z_i^2-(\sum_{i=1}^nZ_i)^2}{n(n-1)}. \end{align*} Squaring leads to\begin{align*} S_n^4=\frac{n^2(\sum_{i=1}^nZ_i^2)^2-2n(\sum_{i=1}^nZ_i^2)(\sum_{i=1}^nZ_i)^2+(\sum_{i=1}^nZ_i)^4}{n^2(n-1)^2} \end{align*} and so\begin{align*} \mathbb{E}(S_n^4)=\frac{n^2\mathbb{E}((\sum_{i=1}^nZ_i^2)^2)-2n\mathbb{E}\left((\sum_{i=1}^nZ_i^2)(\sum_{i=1}^nZ_i)^2 \right)+\mathbb{E}((\sum_{i=1}^n Z_i)^4)}{n^2(n-1)^2}. \end{align*} Since $Z_1,...,Z_n$ are independent, we have that, for distinct $i,j,k$,\begin{align*} \mathbb{E}(Z_iZ_j)=0,\hspace{5mm}\mathbb{E}(Z_i^3Z_j)=0,\hspace{5mm}\mathbb{E}(Z_i^2Z_jZ_k)=0 \end{align*} and\begin{align*} E(Z_i^2Z_j^2)=\mu_2^2=\sigma^4,\hspace{5mm}\mathbb{E}(Z_i^4)=\mu_4. \end{align*} Then, with simple algebraic simplifications, it can be shown that the following holds\begin{align*} \mathbb{E}((\sum_{i=1}^nZ_i^2)^2)&=n\mu_4+n(n-1)\sigma^4,\\ \mathbb{E}\left((\sum_{i=1}^nZ_i^2)(\sum_{i=1}^nZ_i)^2 \right)&=n\mu_4+n(n-1)\sigma^4,\\ \mathbb{E}((\sum_{i=1}^n Z_i)^4)&=n\mu_4+3n(n-1)\sigma^4. \end{align*} Substituting these into the expansion of $\mathbb{E}(S_n^4)$ and simplifying leads to\begin{align*} \mathbb{E}(S_n^4)=\frac{(n-1)\mu_4+(n^2-2n+3)\sigma^4}{n(n-1)} \end{align*} and so\begin{align}\label{var} \mathbb{V}(S_n^2)=\mathbb{E}(S_n^4)-\sigma^4=\frac{(n-1)\mu_4+(n^2-2n+3)\sigma^4}{n(n-1)}-\sigma^4=\frac{\mu_4}{n}-\frac{\sigma^4(n-3)}{n(n-1)}. \end{align}

$\endgroup$
2
  • $\begingroup$ great job! is this correct? $\endgroup$
    – stats_noob
    Jun 3, 2023 at 5:38
  • $\begingroup$ @stats_noob I believe so! $\endgroup$
    – sam wolfe
    Jun 14, 2023 at 12:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .