1
$\begingroup$

An unbiased estimator for a population's variance is:

$$s^2=\frac{1}{n-1}\sum_{i=1}^{n} \left( X_i - \bar{X} \right)^2$$

where $$\bar{X} = \frac{1}{n}\sum_{j=1}^{n} X_j$$

Now, it is widely known that this sample variance estimator is simply consistent (convergence in probability). I wonder, is it also true that it is strongly consistent, i.e. it converges to population variance almost surely? And if yes, are there any additional requirements for $\{X_n\}_{n\geq 1}$?

$\endgroup$
2
$\begingroup$

To begin, we should know under which conditions weak consistency holds. Let's consider the usual case when $X_1,X_2,\ldots$ are i.i.d.r.v.

Since for each $n\in\mathbb N$ $$s^2=\frac1{n-1}\sum_{i=1}^nX_i^2-\frac n{n-1}\bar X^2=\frac n {n-1}\left(\frac1n\sum_{i=1}^nX_i^2-\left(\frac1n\sum_{i=1}^nX_i\right)^2\right).$$

Now, under the hypotheses that allow us to apply the weak or the strong Law of Large Numbers (LLN), we would have $$\frac1n\sum_{i=1}^nX_i\to E(X_1) \quad (1)$$ and $$\frac1n\sum_{i=1}^nX_i^2\to E(X_1^2) \quad (2)$$ ($X_1$ stands for any other variable; it doesn't matter since they all have identical distribution); these limits could mean convergence in probability or almost sure. By the properties of both types of convergence we have $$s^2_n=\frac n {n-1}\left(\frac1n\sum_{i=1}^nX_i^2-\left(\frac1n\sum_{i=1}^nX_i\right)^2\right)\to 1\cdot\left(E(X_1^2)-(E(X_1))^2\right).\quad (3)$$

But it happens that neither $(1)$ or $(2)$ need hold with the assumptions so far mentioned.

Now, $(1)$ is true if $X_i$ has a finite first moment (here we have to assume we have a second moment—otherwise there wouldn't be a variance to estimate); and $(2)$ will hold if $X_i^2$ has finite expectation, which again implies finite second moment for $X_i$ (equivalently, $X_i$ has finite variance $\sigma^2=Var(X_1)$). Say $X_i$ has finite momments up to order two, with $E(X_1^k)=m_k<\infty, \,k\le2$, then $(3)$ becomes $s^2_n\to m_2-m_1^2=\sigma^2,$ and this is true both in probability and almost surely.

So, for i.i.d.v's with finite second momment, $s^2$ is consistent for $\sigma^2$ in strong sense (and so also in weak sense).

If we want an scenario in which consistence is only weak, we may think of a case in which $Var(X_n)=\sigma^2_n$ is not the same for all $n$, but this not only implies that we are not longer assuming identical distributions: in fact, if there is not a unique $\sigma^2$ then there's no point in asking whether $s^2$ is a good estimator for the variance.

Raising independence hypothesis or identical distributions but with a unique variance instead, seems to have less trivial conclusions, but in any case it wouldn't be possible in the case of simple random sampling ($X_n$ i.i.d.).

$\endgroup$
  • $\begingroup$ The Strong Law of Large Numbers does not requires the second moment to be finite. The existence of first moment of i.i.d.r.v's is necessary and sufficient condition for a.s. convergence of $\overline X$ to $E(X_1)$. $\endgroup$ – NCh Feb 6 '18 at 1:48
  • $\begingroup$ What you say is true off course, but I had assumed finite variance in order to have a variance to estimate. It took a while until I found that it wasn't necessary to carry that assumption to the moments of $X_i^2$. Thank you for the remark. $\endgroup$ – Alejandro Nasif Salum Feb 12 '18 at 15:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.