Suppose $X_{1},X_{2},\ldots$ be i.i.d. random variables such that $E\left[X_{i}\right]=\mu$ and $Var(X_{i})=\sigma^{2}<\infty$. Let $\bar{X}=\left(X_{1}+\cdots+X_{n}\right)/n$. Show that $\frac{1}{n}\overset{n}{\underset{i=1}{\sum}}\left(X_{i}-\bar{X}\right)^{2}\rightarrow\sigma^{2}$ a.s.

Solution: Let $S_{n}=\frac{1}{n}\overset{n}{\underset{i=1}{\sum}}\left(X_{i}-\bar{X}\right)^{2}$. By Chebyshev's Inequality, $\sum_{n=1}^{\infty}P\left(\left|S_{n}-E\left[S_{n}\right]\right|>\varepsilon\right)\leq\sum_{n=1}^{\infty}\frac{Var(S_{n})}{\varepsilon^{2}}$. And we know that $E\left[S_{n}\right]\rightarrow\sigma^{2}$, so I would like to finish this proof by using first Borel-Cantelli Lemma if I can show the right hand side is summable. However, $Var(S_{n})=\frac{2\sigma^{4}}{n-1}$, so the sum is infinity, which means it doesn't converge almost surely. Where am I wrong?


Since you used an inequality to obtain an estimate for $\mathbb{P}(|S_n-\mathbb{E}S_n|>\varepsilon)$ the fact that the sum does not converge does not imply that $S_n$ does not converge almost surely.

It's easier like that:

$$\begin{align} (X_i-\mu+(\mu-\bar{X}))^2 &= (X_i-\mu)^2 + 2 (X_i-\mu) \cdot (\mu-\bar{X})+(\mu-\bar{X})^2 \\ \Rightarrow S_n= \frac{1}{n} \sum_{i=1}^n (X_i-\bar{X})^2 &= \frac{1}{n} \sum_{i=1}^n (X_i-\mu)^2 + 2 (\mu-\bar{X}) \underbrace{\frac{1}{n} \cdot \sum_{i=1}^n (X_i-\mu)}_{\left(\frac{1}{n} \sum_{i=1}^n X_i\right)-\mu =(\bar{X}-\mu)} + (\mu-\bar{X})^2 \\ &= \frac{1}{n} \sum_{i=1}^n (X_i-\mu)^2 - (\bar{X}-\mu)^2 \end{align}$$

By the strong law of large numbers we obtain

$$\bar{X} = \frac{1}{n} \sum_{i=1}^n X_i \to \mu \quad \text{almost surely} \\ \frac{1}{n} \sum_{i=1}^n (X_i-\mu)^2 \to \mathbb{E} \left( (X_i-\mu)^2 \right)=\sigma^2 \quad \text{almost surely} $$

Hence $S_n \to \sigma^2$ almost surely.

  • $\begingroup$ But the $\mu$ can be infinity, I think you cannot apply strong law of large numbers. And you haven't used the condition that the variance is finite. $\endgroup$ – 65900931 Nov 23 '12 at 20:28
  • $\begingroup$ Of course I used that $\sigma^2<\infty$ - otherwise I could not apply the Strong Law Of Large Numbers. (And $\mu$ is finite since $\sigma^2$ is finite. $X \in L^2$ implies $X \in L^1$...) $\endgroup$ – saz Nov 23 '12 at 20:41
  • $\begingroup$ That makes sense. Thank you! $\endgroup$ – 65900931 Nov 23 '12 at 20:51

This is much after 2012, but since I chanced upon this, I'd like to just offer an alternative solution (but similar in spirit):

$$\frac{1}{n} \sum_{i=1}^n (X_i-\bar{X})^2 = \frac{1}{n}\sum_{i=1}^n (X_i^2-2X_i\bar{X}+(\bar{X})^2)=\frac{\sum_{i=1}^n X_i^2}{n}-\left(\frac{\sum_{i=1}^n X_i}{n}\right)^2$$

Since $X_i^2$ are i.i.d. with finite mean $\mu^2 + \sigma^2$, by strong law of large numbers (SLLN), $\frac{\sum_{i=1}^n X_i^2}{n} \to \mu^2 + \sigma^2$ almost surely.

In addition, SLLN says $\frac{\sum_{i=1}^n X_i}{n} \to \mu$ almost surely. By Continuous Mapping Theorem (or direct reasoning), $\left(\frac{\sum_{i=1}^n X_i}{n}\right)^2 \to \mu^2$.

Hence, $S_n \to \sigma^2$ almost surely.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.