The central limit theorem establishes that the average of $n$ i.i.d. random variables tends to a normal law, with parameters $\mu$ and $\sigma^2/n$.

The average is a non-biaised estimator of the mean of the distribution. If we turn to the case of the non-biased estimator of the distribution variance, $s^2$, the central limit theorem can also be used as $s^2$ is the difference of

  • the average of $n$ i.i.d. random variables ($\overline{x^2}$),

  • the square of a normally distributed variable (${\overline x}^2$).

Anyway the computation is uneasy as these two variables aren't independent.

As $s^2$ is unbiaised, its mean is $\sigma^2$. But what is the variance of $s^2$ ? And is its distribution normal in the limit ?

  • 1
    $\begingroup$ You are considering $$\bar x=\frac1n\sum_{i=1}^nx_i\qquad s^2=\frac1n\left(\sum_{i=1}^nx_i^2\right)-\bar x^2$$ then $$E(s^2)=\frac{n-1}n\sigma^2$$ hence $s^2$ is a biased estimator of the variance $\sigma^2=\mathrm{var}(x_i^2)$. $\endgroup$ – Did Jun 16 '17 at 21:05
  • $\begingroup$ @Did: I am unsure about the notation and I want to refer to the unbiaised estimator. Anyway, this distinction is not essential as they are proportional to each other. $\endgroup$ – Yves Daoust Jun 16 '17 at 21:07
  • 1
    $\begingroup$ Then correct your post. And what prevents you to compute $E(s^4)$? $\endgroup$ – Did Jun 16 '17 at 21:08
  • $\begingroup$ @Did: no idea of the relation between $E(s^4)$, presumably drawn from the moments of the original distribution, and the variance of the limit distribution of $s^2$ (which I don't know). $\endgroup$ – Yves Daoust Jun 16 '17 at 21:14
  • $\begingroup$ I disagree with the downvote. $\endgroup$ – Yves Daoust Jun 18 '17 at 13:46

Let $s^2=\frac{n}{n-1}\left(\overline{x^2}-\left(\overline x\right)^2\right)=\frac{1}{n-1}\sum_{i=1}^n \left(x_i-\overline x\right)^2$. Define centered r.v.'s $y_i=x_i-\mathbb Ex_1$ and rewrite sample variance in terms of this r.v.'s: $$ s^2=\frac{1}{n-1}\sum_{i=1}^n \left(y_i-\overline y\right)^2 = \frac{n}{n-1}\left(\overline{y^2}-\left(\overline y\right)^2\right)=\overline{y^2}-\left(\overline y\right)^2+\frac{s^2}{n}. $$ Note that $\sigma^2=\text{Var}(x_1)=\mathbb E[y_1^2]$.

Find the limiting distribution of $\sqrt{n}\left(s^2-\sigma^2\right)$: $$\tag{1}\label{1} \sqrt{n}\left(s^2-\sigma^2\right) = \sqrt{n}\left(\overline{y^2}-\left(\overline y\right)^2+\frac{s^2}{n} -\sigma^2 \right)=\sqrt{n}\left(\overline{y^2}-\sigma^2 \right) -\sqrt{n}\left(\overline y\right)^2+\sqrt{n}\frac{s^2}{n} . $$ Next prove that $\sqrt{n}\left(\overline y\right)^2 \xrightarrow{p} 0$ and $\sqrt{n}\,\dfrac{s^2}{n}=\dfrac{s^2}{\sqrt{n}}\xrightarrow{p} 0$ as $n\to\infty$. Indeed, by Slutsky's theorem, $$\sqrt{n}\left(\overline y\right)^2 = \underbrace{\overline y}_{\begin{array}$\downarrow p \cr 0\end{array}} \cdot \underbrace{\sqrt{n}\left(\overline y\right)}_{\begin{array}$\downarrow d \cr N(0,1)\end{array}}\xrightarrow{d} 0\cdot N(0,1)=0$$ The convergence in distribution to zero implies the convergence in probability.

Next, $$ \dfrac{s^2}{\sqrt{n}} = s^2\cdot \frac{1}{\sqrt{n}}\xrightarrow{p} \sigma^2\cdot 0=0.$$ We obtain that the second and third terms in r.h.s. of (\ref{1}) tends to zero in probability. Consider the first term: $$ \sqrt{n}\left(\overline{y^2}-\sigma^2 \right) = \sqrt{n}\left(\overline{y^2}-\mathbb E\left[y_1^2\right] \right) \xrightarrow{d} N(0,\text{Var}(y_1^2))=N(0,\mathbb E\left[y_1^4\right]-\sigma^4). $$ By Slutsky's theorem, $$\tag{2}\label{2} \sqrt{n}(s^2-\sigma^2)\xrightarrow{d}N(0,\mathbb E\left[y_1^4\right]-\sigma^4)= N(0,{\mathbb E}\left[(x_1-\mathbb Ex_1)^4\right]-\sigma^4). $$ So, you can say that the limiting distribution of $s^2$ is normal with mean $\sigma^2$ and variance $$\dfrac{{\mathbb E}\left[(x_1-\mathbb Ex_1)^4\right]-\sigma^4}{n}.$$ But this words are extremely non-rigorous. The rigorous statement is (\ref{2}).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.