I'm studying about Markov Processes and I came across the following exercise in my reference book (Daniel W. Stroock An Introduction to Markov Processes):

Let $\{Y_n:n\geq 1\}$ be a sequence of mutually independent, identically distributed random variables satisfying $E[Y_1]<\infty$. Set $X_n=\sum_{m=1}^n Y_m$ for $n\geq 1$. The Weak Law of Large Numbers says that

$$P\left(\left|\frac{X_n}{n}-E[Y_1]\right|\geq \epsilon\right)\rightarrow 0\;\;\;\text{for all } \epsilon>0.$$

In fact,

$$\lim_{n\rightarrow \infty} E\left[\left|\frac{X_n}{n}-E[Y_1]\right|\right]=0,\;\;\;\;\;\;(1.3.3)$$

from which the above follows as an application of Markov's inequality. Here are steps which lead to (1.3.3).

(a) First reduce to the case when $E[Y_1]=0$. Next, asume that $E[Y_1^2]<\infty$, and show that

$$E\left[\left|\frac{X_n}{n}\right|\right]^2 \leq E\left[\left|\frac{X_n}{n}\right|^2\right]=\frac{E[Y_1^2]}{n}.$$

Hence the result is proved when $Y_1$ has a finite second moment.

How do I go about solving task (a)? I tried to simplify the inequality in (a):

$$E\left[\left|\frac{X_n}{n}\right|\right]^2 \leq E\left[\left|\frac{X_n}{n}\right|^2\right]\Leftrightarrow E\left[\left|X_n\right|\right]^2 \leq E\left[\left|X_n\right|^2\right],$$

But at this point I got stuck, what am I missing here now? Thank you for any help!

  • $\begingroup$ Are you asking why $E(X)^2\leqslant E(X^2)$? This is Cauchy-Schwarz... $\endgroup$ – Did Jul 29 '17 at 10:57
  • $\begingroup$ Thank you @Did. Yes and also about the rest, i.e. why $E\left[\left|\frac{X_n}{n}\right|^2\right]=\frac{E[Y_1^2]}{n}$. I was not aware of this analogous inequality in probability theory :) $\endgroup$ – jjepsuomi Jul 29 '17 at 11:04
  • $\begingroup$ The next step is to note that the variance of a sum of independent random variables is the sum of their variances. Please note that these are very basic arguments, present in every proof related to the law of large numbers. $\endgroup$ – Did Jul 29 '17 at 12:01
  • $\begingroup$ Thank you @Did I answered the question myself now :) I understand, I don't do this so regularly so I forget some formulas now and then. $\endgroup$ – jjepsuomi Jul 29 '17 at 12:18

I think I got it now myself. Thank you to @Did for noting the Cauchy-Schwarz :) I need to show first that$$E\left[\left|\frac{X_n}{n}\right|\right]^2 = \frac{1}{n^2}E\left[|X_n|\right]^2 \leq \frac{1}{n^2}E\left[|X_n|^2\right] =E\left[\left|\frac{X_n}{n}\right|^2\right],$$

which results immediately from Cauchy-Schwarz:

$$\lvert E[XY]\rvert \leq \sqrt{E[X^2]} \sqrt{E[Y^2]},$$

by setting $X=1, Y=|X_n|$ and squaring both sides. Next I show that $ E\left[\left|\frac{X_n}{n}\right|^2\right]=\frac{E[Y_1^2]}{n}.$

$$E\left[\left|\frac{X_n}{n}\right|^2\right]=E\left[\frac{X_n^2}{n^2}\right]=\frac{1}{n^2}E[X_n^2] = \frac{1}{n^2}E\left[\sum_{m=1}^n Y_m\sum_{m=1}^n Y_m\right] = \frac{1}{n^2}E\left[\sum_{m=1}^n Y_m^2+2\sum_{i\neq j} Y_iY_j\right]$$

$$=\frac{n}{n^2}E\left[Y_1^2\right]+2\sum_{i \neq j}E[Y_i]E[Y_j]=\frac{n}{n^2}E\left[Y_1^2\right] + 0 = \frac{E[Y_1^2]}{n},$$

which is what was to be shown $\square$. I used the fact that $Y_i, Y_j$ were i.i.d. for $i\neq j$ and $E[Y_1]=0$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.