Sample variance converge almost surely

Question

Suppose $X_{1},X_{2},\ldots$ be i.i.d. random variables such that $E\left[X_{i}\right]=\mu$ and $Var(X_{i})=\sigma^{2}<\infty$. Let $\bar{X}=\left(X_{1}+\cdots+X_{n}\right)/n$. Show that $\frac{1}{n}\overset{n}{\underset{i=1}{\sum}}\left(X_{i}-\bar{X}\right)^{2}\rightarrow\sigma^{2}$ a.s.

Solution: Let $S_{n}=\frac{1}{n}\overset{n}{\underset{i=1}{\sum}}\left(X_{i}-\bar{X}\right)^{2}$. By Chebyshev's Inequality, $\sum_{n=1}^{\infty}P\left(\left|S_{n}-E\left[S_{n}\right]\right|>\varepsilon\right)\leq\sum_{n=1}^{\infty}\frac{Var(S_{n})}{\varepsilon^{2}}$. And we know that $E\left[S_{n}\right]\rightarrow\sigma^{2}$, so I would like to finish this proof by using first Borel-Cantelli Lemma if I can show the right hand side is summable. However, $Var(S_{n})=\frac{2\sigma^{4}}{n-1}$, so the sum is infinity, which means it doesn't converge almost surely. Where am I wrong?

Answer 1

Since you used an inequality to obtain an estimate for $\mathbb{P}(|S_n-\mathbb{E}S_n|>\varepsilon)$ the fact that the sum does not converge does not imply that $S_n$ does not converge almost surely.

It's easier like that:

$$\begin{align} (X_i-\mu+(\mu-\bar{X}))^2 &= (X_i-\mu)^2 + 2 (X_i-\mu) \cdot (\mu-\bar{X})+(\mu-\bar{X})^2 \\ \Rightarrow S_n= \frac{1}{n} \sum_{i=1}^n (X_i-\bar{X})^2 &= \frac{1}{n} \sum_{i=1}^n (X_i-\mu)^2 + 2 (\mu-\bar{X}) \underbrace{\frac{1}{n} \cdot \sum_{i=1}^n (X_i-\mu)}_{\left(\frac{1}{n} \sum_{i=1}^n X_i\right)-\mu =(\bar{X}-\mu)} + (\mu-\bar{X})^2 \\ &= \frac{1}{n} \sum_{i=1}^n (X_i-\mu)^2 - (\bar{X}-\mu)^2 \end{align}$$

By the strong law of large numbers we obtain

$$\bar{X} = \frac{1}{n} \sum_{i=1}^n X_i \to \mu \quad \text{almost surely} \\ \frac{1}{n} \sum_{i=1}^n (X_i-\mu)^2 \to \mathbb{E} \left( (X_i-\mu)^2 \right)=\sigma^2 \quad \text{almost surely} $$

Hence $S_n \to \sigma^2$ almost surely.

Answer 2

This is much after 2012, but since I chanced upon this, I'd like to just offer an alternative solution (but similar in spirit):

$$\frac{1}{n} \sum_{i=1}^n (X_i-\bar{X})^2 = \frac{1}{n}\sum_{i=1}^n (X_i^2-2X_i\bar{X}+(\bar{X})^2)=\frac{\sum_{i=1}^n X_i^2}{n}-\left(\frac{\sum_{i=1}^n X_i}{n}\right)^2$$

Since $X_i^2$ are i.i.d. with finite mean $\mu^2 + \sigma^2$, by strong law of large numbers (SLLN), $\frac{\sum_{i=1}^n X_i^2}{n} \to \mu^2 + \sigma^2$ almost surely.

In addition, SLLN says $\frac{\sum_{i=1}^n X_i}{n} \to \mu$ almost surely. By Continuous Mapping Theorem (or direct reasoning), $\left(\frac{\sum_{i=1}^n X_i}{n}\right)^2 \to \mu^2$.

Hence, $S_n \to \sigma^2$ almost surely.