2
$\begingroup$

The central limit theorem establishes that the average of $n$ i.i.d. random variables tends to a normal law, with parameters $\mu$ and $\sigma^2/n$.

The average is a non-biaised estimator of the mean of the distribution. If we turn to the case of the non-biased estimator of the distribution variance, $s^2$, the central limit theorem can also be used as $s^2$ is the difference of

  • the average of $n$ i.i.d. random variables ($\overline{x^2}$),

  • the square of a normally distributed variable (${\overline x}^2$).

Anyway the computation is uneasy as these two variables aren't independent.

As $s^2$ is unbiaised, its mean is $\sigma^2$. But what is the variance of $s^2$ ? And is its distribution normal in the limit ?

$\endgroup$
5
  • 1
    $\begingroup$ You are considering $$\bar x=\frac1n\sum_{i=1}^nx_i\qquad s^2=\frac1n\left(\sum_{i=1}^nx_i^2\right)-\bar x^2$$ then $$E(s^2)=\frac{n-1}n\sigma^2$$ hence $s^2$ is a biased estimator of the variance $\sigma^2=\mathrm{var}(x_i^2)$. $\endgroup$
    – Did
    Commented Jun 16, 2017 at 21:05
  • $\begingroup$ @Did: I am unsure about the notation and I want to refer to the unbiaised estimator. Anyway, this distinction is not essential as they are proportional to each other. $\endgroup$
    – user65203
    Commented Jun 16, 2017 at 21:07
  • 1
    $\begingroup$ Then correct your post. And what prevents you to compute $E(s^4)$? $\endgroup$
    – Did
    Commented Jun 16, 2017 at 21:08
  • $\begingroup$ @Did: no idea of the relation between $E(s^4)$, presumably drawn from the moments of the original distribution, and the variance of the limit distribution of $s^2$ (which I don't know). $\endgroup$
    – user65203
    Commented Jun 16, 2017 at 21:14
  • $\begingroup$ I disagree with the downvote. $\endgroup$
    – user65203
    Commented Jun 18, 2017 at 13:46

1 Answer 1

3
$\begingroup$

Let $s^2=\frac{n}{n-1}\left(\overline{x^2}-\left(\overline x\right)^2\right)=\frac{1}{n-1}\sum_{i=1}^n \left(x_i-\overline x\right)^2$. Define centered r.v.'s $y_i=x_i-\mathbb Ex_1$ and rewrite sample variance in terms of this r.v.'s: $$ s^2=\frac{1}{n-1}\sum_{i=1}^n \left(y_i-\overline y\right)^2 = \frac{n}{n-1}\left(\overline{y^2}-\left(\overline y\right)^2\right)=\overline{y^2}-\left(\overline y\right)^2+\frac{s^2}{n}. $$ Note that $\sigma^2=\text{Var}(x_1)=\mathbb E[y_1^2]$.

Find the limiting distribution of $\sqrt{n}\left(s^2-\sigma^2\right)$: $$\tag{1}\label{1} \sqrt{n}\left(s^2-\sigma^2\right) = \sqrt{n}\left(\overline{y^2}-\left(\overline y\right)^2+\frac{s^2}{n} -\sigma^2 \right)=\sqrt{n}\left(\overline{y^2}-\sigma^2 \right) -\sqrt{n}\left(\overline y\right)^2+\sqrt{n}\frac{s^2}{n} . $$ Next prove that $\sqrt{n}\left(\overline y\right)^2 \xrightarrow{p} 0$ and $\sqrt{n}\,\dfrac{s^2}{n}=\dfrac{s^2}{\sqrt{n}}\xrightarrow{p} 0$ as $n\to\infty$. Indeed, by Slutsky's theorem, $$\sqrt{n}\left(\overline y\right)^2 = \underbrace{\overline y}_{\begin{array}$\downarrow p \cr 0\end{array}} \cdot \underbrace{\sqrt{n}\left(\overline y\right)}_{\begin{array}$\downarrow d \cr N(0,1)\end{array}}\xrightarrow{d} 0\cdot N(0,1)=0$$ The convergence in distribution to zero implies the convergence in probability.

Next, $$ \dfrac{s^2}{\sqrt{n}} = s^2\cdot \frac{1}{\sqrt{n}}\xrightarrow{p} \sigma^2\cdot 0=0.$$ We obtain that the second and third terms in r.h.s. of (\ref{1}) tends to zero in probability. Consider the first term: $$ \sqrt{n}\left(\overline{y^2}-\sigma^2 \right) = \sqrt{n}\left(\overline{y^2}-\mathbb E\left[y_1^2\right] \right) \xrightarrow{d} N(0,\text{Var}(y_1^2))=N(0,\mathbb E\left[y_1^4\right]-\sigma^4). $$ By Slutsky's theorem, $$\tag{2}\label{2} \sqrt{n}(s^2-\sigma^2)\xrightarrow{d}N(0,\mathbb E\left[y_1^4\right]-\sigma^4)= N(0,{\mathbb E}\left[(x_1-\mathbb Ex_1)^4\right]-\sigma^4). $$ So, you can say that the limiting distribution of $s^2$ is normal with mean $\sigma^2$ and variance $$\dfrac{{\mathbb E}\left[(x_1-\mathbb Ex_1)^4\right]-\sigma^4}{n}.$$ But this words are extremely non-rigorous. The rigorous statement is (\ref{2}).

$\endgroup$

You must log in to answer this question.