# Convergence in Probability for sum of $X_i^2$

So I know that I have to find $\textbf{E}[Y_n]$ and $\text{var}(Y_n)$ and then use Chebyshev's inequality to prove this question, but I'm having trouble deriving $\textbf{E}(Y_n)$ and $\text{var}(Y_n)$. I think I got $\textbf{E}(Y_n)$, which is $$\textbf{E}(X_i^2) = \text{var}(X_i) + (\textbf{E}(X_i))^2.$$

Hopefully this is correct. I do not know how to get $\text{var}(Y_n)$ after this though as it requires finding $\textbf{E}(Y_n^2)$.

I think end result should look something like $$P(|Y_n-\textbf{E}(Y_n)|>\epsilon) < \frac{\text{var}(Y_n)}{\epsilon^2}.$$

And then obviously I would write $\textbf{E}(Y_n)$ and $\text{var}(Y_n)$ in terms of $\textbf{E}(X_1)$ and $\text{var}(X_1)$ as I am asked to do so in the question, and finally prove that the right hand side = $0$ as $n \rightarrow \infty$.

• Hint: For $var(Y_n)$ use the facts that functions of IID random variables are still IID random variables and if $X$ and $Y$ are two independent random variables then we have $var(X+Y)=var(X)+var(Y)$. – rookie Feb 16 '17 at 8:29

I think this is the easiest approach

We want to apply the strong law, which says that for i.i.d. $\xi_n$, $\frac{1}{n}\sum \xi_n \rightarrow E\xi_1$ a.s. as long as $E|\xi_1| < \infty$. Now let $\xi_n = X_n^2$, we can apply the law of large numbers as long as$EX_1^2 < \infty$. However, we know that $\text{Var}(X_1) = EX_1^2 - (EX_1)^2$ and therefore $EX_1^2 = \text{Var}(X_1) + (EX_1)^2 < \infty$ and we can apply the law of large numbers to see that $\frac{1}{n} \sum X_n^2 \rightarrow EX_1^2$ a.s., but we already know the that $EX_1^2 = \text{Var}(X_1) + (EX_1)^2$ and we are done

Indeed, you made use of the linearity of expectation:

\begin{align}\mathsf {E}(Y_n) ~&=~ \mathsf {E}(\sum_{j=1}^n\tfrac 1nX_j^2) \\[1ex] ~&=~ \tfrac 1n\sum\limits_{i=1}^n\mathsf {E}(X_i^2)\end{align}

Then you called upon the identical distribution of the $X_i$, and the definition of variance:

\begin{align}\mathsf {E}(Y_n) ~&=~ \mathsf {E}(X_1^2) \\[1ex] &=~ \mathsf {Var}(X_1)+\mathsf E(X_1)^2\end{align}

Now using mostly the same approach, we can apply the bilinearity of covariance:

\begin{align}\mathsf {Var}(Y_n) ~&=~ \mathsf {Cov}(\sum_{j=1}^n\tfrac 1nX_j^2,\sum_{k=1}^n\tfrac 1nX_k^2) \\[1ex] ~&=~ \sum\limits_{i=1}^n\mathsf {Var}( \tfrac 1nX_i^2)+\underset{1\leq j<k\leq n}{2\sum}\!\mathsf {Cov}(\tfrac 1n X_j^2,\tfrac 1nX_k^2)\end{align}

And of course, since the random variables are independent, as well as identically distributed, this is simplified:

\begin{align}\mathsf {Var}(Y_n) ~&=~ \sum\limits_{i=1}^n\mathsf {Var}( \tfrac 1nX_i^2) \\[1ex] & =~ \tfrac 1n\mathsf {Var}(X_1^2)\end{align}

That however, will not be obtainable in terms of $\mathsf E(X_1)$, $\mathsf{Var}(X_1)$.   Though you should be able to make claims about the limit as $n\to\infty$.

• I think you mean that $E[Y_n]=E[X_1^2]=Var[X_1]+E[X_1]^2$ – Cuoredicervo Feb 16 '17 at 8:40
• Cool. I see that since the variance of Yn is a fraction over n, the limit as n approaches infinity should then be 0. I think this is enough to answer the question. – Lebron James Feb 16 '17 at 8:50
• @Adam Since $\text{Var} (X_1^2)$ depends on the fourth moment of $X_1$ one cannot say that $\text{Var} (X_1^2)$ is finite. – rookie Feb 16 '17 at 9:06

For evaluating $var(Y_n)$ refer my comment in the question or refer the other answer.

I write this answer to prove that $Y_n \overset{p}{\longrightarrow} E(Y_n)$ only with the hypothesis that $E(X_i^2)$ is finite. The proof by Chebyshev inequality uses the fact that $\text{Var} (X_1^2) < \infty$ which is not true in general and by the given hypothesis. So following is the proof using the characteristic function.

By Taylor's theorem for complex functions, the characteristic function of any random variable, $V$, with finite mean $\mu$, can be written as:

$\varphi_V(t) = 1 + it\mu + o(t), \quad t \rightarrow 0.$

All $X_1^2,X_2^2,\cdots$ have the same characteristic function, so we will simply denote this by $\varphi_V$.

Among the basic properties of characteristic functions there are: $\varphi_{\frac 1 n X}(t)= \varphi_X(\tfrac t n) \quad \text{and} \quad \varphi_{X+Y}(t)=\varphi_X(t) \varphi_Y(t) \quad$ if $X$ and $Y$ are independent.

These rules can be used to calculate the characteristic function of $Y_n$ in terms of $\varphi_V$:

$\varphi_{Y_n}(t)= \left[\varphi_V\left({t \over n}\right)\right]^n = \left[1 + i\mu{t \over n} + o\left({t \over n}\right)\right]^n \, \rightarrow \, e^{it\mu}, \quad \text{as} \quad n \rightarrow \infty.$

The limit $e^{it\mu}$ is the characteristic function of the constant random variable $\mu$, and hence by the Lévy continuity theorem, $Y_n$ converges in distribution to $\mu$.

Since $\mu$ is a constant, convergence in distribution to $\mu$ and convergence in probability to $\mu$ are equivalent.

Therefore, $Y_n \overset{p}{\longrightarrow} E(Y_n)$ whenever $E(X_i^2)$ is finite.

And $E(Y_n) = E(X_1^2) = var(X_1) + (E(X_1))^2.$

• Thank you. The question assumes that both var(x) and E(x) are finite, so E(x^2) I would assume must be finite as it is the addition of var(x) and (E(x))^2, two finite numbers. – Lebron James Feb 16 '17 at 18:19
• Glad to be of help. Yes and thus this proof is a valid one since it uses finiteness of only $E(X^2)$ and not of $E(X^4)$ which is essential when Chebyshev's inequality is used. – rookie Feb 17 '17 at 3:22
• Why don't just use strong law of large numbers, which holds as long as each element of the sequence is integrable? – Ran Wang Feb 20 '17 at 22:11
• @Ran Wang: Yes. Strong law of large numbers is applicable. I just wanted to complete the proof without the use of SLLN directly. – rookie Feb 21 '17 at 4:05