# Proving The Weak Law of Large Numbers

I'm studying about Markov Processes and I came across the following exercise in my reference book (Daniel W. Stroock An Introduction to Markov Processes):

Let $\{Y_n:n\geq 1\}$ be a sequence of mutually independent, identically distributed random variables satisfying $E[Y_1]<\infty$. Set $X_n=\sum_{m=1}^n Y_m$ for $n\geq 1$. The Weak Law of Large Numbers says that

$$P\left(\left|\frac{X_n}{n}-E[Y_1]\right|\geq \epsilon\right)\rightarrow 0\;\;\;\text{for all } \epsilon>0.$$

In fact,

$$\lim_{n\rightarrow \infty} E\left[\left|\frac{X_n}{n}-E[Y_1]\right|\right]=0,\;\;\;\;\;\;(1.3.3)$$

from which the above follows as an application of Markov's inequality. Here are steps which lead to (1.3.3).

(a) First reduce to the case when $E[Y_1]=0$. Next, asume that $E[Y_1^2]<\infty$, and show that

$$E\left[\left|\frac{X_n}{n}\right|\right]^2 \leq E\left[\left|\frac{X_n}{n}\right|^2\right]=\frac{E[Y_1^2]}{n}.$$

Hence the result is proved when $Y_1$ has a finite second moment.

How do I go about solving task (a)? I tried to simplify the inequality in (a):

$$E\left[\left|\frac{X_n}{n}\right|\right]^2 \leq E\left[\left|\frac{X_n}{n}\right|^2\right]\Leftrightarrow E\left[\left|X_n\right|\right]^2 \leq E\left[\left|X_n\right|^2\right],$$

But at this point I got stuck, what am I missing here now? Thank you for any help!

• Are you asking why $E(X)^2\leqslant E(X^2)$? This is Cauchy-Schwarz... – Did Jul 29 '17 at 10:57
• Thank you @Did. Yes and also about the rest, i.e. why $E\left[\left|\frac{X_n}{n}\right|^2\right]=\frac{E[Y_1^2]}{n}$. I was not aware of this analogous inequality in probability theory :) – jjepsuomi Jul 29 '17 at 11:04
• The next step is to note that the variance of a sum of independent random variables is the sum of their variances. Please note that these are very basic arguments, present in every proof related to the law of large numbers. – Did Jul 29 '17 at 12:01
• Thank you @Did I answered the question myself now :) I understand, I don't do this so regularly so I forget some formulas now and then. – jjepsuomi Jul 29 '17 at 12:18

I think I got it now myself. Thank you to @Did for noting the Cauchy-Schwarz :) I need to show first that$$E\left[\left|\frac{X_n}{n}\right|\right]^2 = \frac{1}{n^2}E\left[|X_n|\right]^2 \leq \frac{1}{n^2}E\left[|X_n|^2\right] =E\left[\left|\frac{X_n}{n}\right|^2\right],$$

which results immediately from Cauchy-Schwarz:

$$\lvert E[XY]\rvert \leq \sqrt{E[X^2]} \sqrt{E[Y^2]},$$

by setting $X=1, Y=|X_n|$ and squaring both sides. Next I show that $E\left[\left|\frac{X_n}{n}\right|^2\right]=\frac{E[Y_1^2]}{n}.$

$$E\left[\left|\frac{X_n}{n}\right|^2\right]=E\left[\frac{X_n^2}{n^2}\right]=\frac{1}{n^2}E[X_n^2] = \frac{1}{n^2}E\left[\sum_{m=1}^n Y_m\sum_{m=1}^n Y_m\right] = \frac{1}{n^2}E\left[\sum_{m=1}^n Y_m^2+2\sum_{i\neq j} Y_iY_j\right]$$

$$=\frac{n}{n^2}E\left[Y_1^2\right]+2\sum_{i \neq j}E[Y_i]E[Y_j]=\frac{n}{n^2}E\left[Y_1^2\right] + 0 = \frac{E[Y_1^2]}{n},$$

which is what was to be shown $\square$. I used the fact that $Y_i, Y_j$ were i.i.d. for $i\neq j$ and $E[Y_1]=0$.