During one of our Information theory classes, the professor constructed the following set:

$$T_\delta = \left\{\mathbf{y} \in \mathbb{R}^n: \frac{\sum_{i=1}^ny_i^2}{n} \leq P + \sigma^2+\delta\right\}$$ where $\delta > 0$.

He mentioned that if $x_n$ is a real valued sequence such that, $\sum_{i=1}^nx_i^2 \leq nP$, $Z_i$ are iid normal distributed with zero mean and variance $\sigma^2$, $Y_i = x_i + Z_i$, then it follows by WLLN (weak law of large numbers) that

$$ \mathbb{P}\left[ \mathbf{Y} \in T_\delta\right]\to 1 ~ \mbox{as}~ n\to \infty$$

Here $\mathbf{Y}$ is $Y_1,Y_2,\cdots,Y_n$ taken as a vector in $\mathbb{R}^n$.

I felt that I could prove the result via strong laws, so this is what I got:

$$\frac{\sum_{i=1}^n Y_i^2}{n} = \frac{\sum_{i=1}^n x_i^2 + Z_i^2 + 2x_iZ_i}{n}$$ As $n\to \infty$, $\lim \frac{\sum_{i=1}^n x_i^2}{n} \leq P$ and $\lim \frac{\sum_{i=1}^n Z_i^2}{n} = \sigma^2 ~ \mbox{a.s}$.

However, I could not tackle the last term. If you apply Cauchy Schwarz, you get $$\frac{\sum_{i=1}^nx_iZ_i}{n} \leq \sqrt{P\sigma^2}$$

But then the net upper bound evaluates to $(\sqrt{P}+\sigma)^2$ which is weaker than the desired bound.

I'd appreciate any ideas on this. Feel free to request clarifications if necessary. If you suspect something stronger is required here, let me know.

Update: I asked my Prof. about this. He said that he didn't know if it held in the strong sense but the weak sense would be provable. What he did was first show $\sum_{i=1}^nE[Y_i^2] \leq n(P+\sigma^2)$ which follows by expansion and substitution. From this, it follows that:

$$\mathbb{P}\left[\frac{\sum_{i=1}^nY_i^2}{n} \leq \frac{\sum_{i=1}^nE[Y_i^2]}{n}+\delta\right] \leq \mathbb{P}\left[\frac{\sum_{i=1}^nY_i^2}{n} \leq P + \sigma^2+\delta\right]$$

Hence it suffices to show $$\mathbb{P}\left[\frac{\sum_{i=1}^n(Y_i^2-E[Y_i^2])}{n} \leq \delta\right] \to 1$$ I was able to get the rest with the following: From Chebyshev's inequality, $$\mathbb{P}\left[\frac{\sum_{i=1}^n(Y_i^2-E[Y_i^2])}{n} \leq \delta\right] \geq 1 - \frac{\sum_{i=1}^nVar(Y_i^2)}{n^2\delta^2}$$ $$= 1 - \frac{\sum_{i=1}^n Var(Z_i^2+2x_iZ_i)}{n^2\delta^2}$$ Note that I've used that $Var(X+c) = Var(X)$ where $c$ is a constant. Now expanding this yields $$Var(Z_i^2+2x_iZ_i) = 2\sigma^4 +4x_i^2\sigma^2 $$ which yields $$\sum_{i=1}^nVar(Z_i^2+2x_iZ_i) \leq 2n\sigma^4 +4\sum_{i=1}^nx_i^2\sigma^2\leq n(2\sigma^4+4P\sigma^2) $$

From this the "weak" result follows. However, my question is whether the stronger result holds? Update: I mentioned weak and strong but I forgot to elaborate. Essentially by strong result what I mean is can I conclude $$\lim_{n\to \infty}\frac{\sum_{i=1}^nY_i^2}{n} \leq P + \sigma^2+\delta ~ \mbox{a.s}$$ What I proved was a convergence in probability type result i.e. Weak Law type result.


For the strong result: $$\frac 1n\sum_{i=1}^nY_i^2=\frac 1n\sum_{i=1}^nx_i^2+\frac 2n\sum_{i=1}^nx_iZ_i+\frac 1n\sum_{i=1}^nZ_i^2\leqslant P+\frac 2n\sum_{i=1}^nx_iZ_i+\frac 1n\sum_{i=1}^nZ_i^2,$$ so it suffices to show that almost surely, $$\tag{*}\lim_{n\to +\infty}\frac 2n\sum_{i=1}^nx_iZ_i+\frac 1n\sum_{i=1}^nZ_i^2 \leqslant \delta+\sigma^2.$$ Since the sequence $(Z_i^2)_{i\geqslant 1}$ is independent and identically distributed, $\sum_{i=1}^nZ_i^2/n\to \sigma^2$ almost surely, (*) reduces to $$\lim_{n\to +\infty}\frac 2n\sum_{i=1}^nx_iZ_i\leqslant \delta.$$ Since $\sum_{i=1}^nx_iZ_i$ is Gaussian, centered and its variance is $\sum_{i=1}^nx_i^2$, it has the same distribution as $\left(\sum_{i=1}^nx_i^2\right)^{1/2}N$, where $N$ is standard normal. Therefore, $$\mathbb E\left|\frac 2n\sum_{i=1}^nx_iZ_i\right|^4 =\left(\frac 2n\right)^4\left(\sum_{i=1}^nx_i^2\right)^2\mathbb E|N|^4,$$

and using the assumption on the $x_i$, Markov's inequality and the Borel-Cantelli lemma, we deduce that $\sum_{i=1}^nx_iZ_i/n\to 0$ almost surely.

  • $\begingroup$ Thanks for the elegant solution Davide... It's good to know that it held in strong sense as well. $\endgroup$ – Gautam Shenoy Dec 30 '15 at 15:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.