# Poke holes in this proof of the SLLN

I have a proof (sketch) of the Strong Law of Large Numbers, at least the "sufficiency" half of it, that seems a little too easy. This is the version where you only assume i.i.d. random variables, and $$E[X]<\infty$$.

The idea is to derive it ultimately from the special case of i.i.d. Bernoulli r.v.s. That case is relatively elementary to prove: you can get it from the convergence of the binomial to the normal, for instance, or by showing that the 4th centralized moment is proportional to $$n^2$$ for the binomial distribution. The apply Borel-Cantelli, etc.

The first step is to approximate the random variable X's by discrete Y's (if they aren't already discrete). This can be done in such a way that $$E[X] = E[Y] = \mu$$. Simply partition the real numbers, and then compute the expectation of X on each set in the partition. Then when $$X = x$$ occurs, $$x$$ is in one of the partitioning sets, say $$B_i$$. Then we say that $$Y = y$$ has occurred, where $$y$$ is the average of $$X$$ on that partitioning set. In other words,

$$Y = \sum_i E[X|X \in B_i]1_{\{X \in B_i\}}$$

That way, the discrepancy between $$X$$ and $$Y$$ becomes smaller as the partition becomes more refined. If the partitioning sets are intervals, then $$|X-Y| \leq \Delta$$, where $$\Delta$$ is the width of the largest interval in the partition. More importantly,

$$|\overline{X}_n - \overline{Y}_n| \leq \Delta$$

Having approximated $$X$$ by a discrete r.v. $$Y$$, and argued that their sample means can be made arbitrarily close, it remains to show that the SLLN holds for discrete r.v.s with finite mean. If $$Y$$ takes values $$y_1, y_2, y_3,...$$ with probabilities $$p_1, p_2, p_3,...$$ then you can say

$$Y = \sum_i y_i Z_i \quad \mbox{where} \quad Z_i = 1_{ Y^{-1}(y_i)}$$

In other words, you can look at it as a weighted sum of Bernoulli r.v.s with respective success probabilities $$p_1, p_2,$$, etc. For each of these Bernoulli random variables $$Z_i$$ (i.e. success if $$Y=y_1$$, failure otherwise), the SLLN for Bernoulli sequences tells us that

$$\overline{Z}_{i,n} - p_i \xrightarrow{a.s.} 0$$

Since we are assuming that $$\mu = \sum_i p_i y_i < \infty$$, then for any $$\delta > 0$$ we can choose $$I$$ large enough so that $$\sum_{i>I} p_i y_i < \delta$$. Say we have some $$\delta$$ and its corresponding $$I$$ in mind. By linearity of almost sure convergence,

$$\sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^I p_iy_i \xrightarrow{a.s.} 0$$

In other words, for any $$\epsilon > 0$$

$$P\left( \left| \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^I p_iy_i \right| > \epsilon \quad i.o. \right) = 0$$

By assumption, $$\left| \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^I p_iy_i \right| > \left| \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^\infty p_iy_i \right| - \delta$$. So, for each pair $$(\delta, I)$$

$$P\left( \left| \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^\infty p_iy_i \right| > \epsilon + \delta \quad i.o. \right) \leq P\left( \left| \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^I p_iy_i \right| > \epsilon \quad i.o. \right) = 0$$

That is, $$P\left( \left| \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^\infty p_iy_i \right| > \epsilon + \delta \quad i.o. \right) = 0$$ for each $$(\delta, I)$$. Since $$I$$ gets large as $$\delta$$ gets small, I'm thinking this means that

$$P\left( \left| \sum_{i=1}^\infty y_i\overline{Z}_{i,n} - \sum_{i=1}^\infty p_iy_i \right| > \epsilon \quad i.o. \right) = 0$$

(This seems "obvious", but it's the step I'm not sure exactly how to justify if it needs it. Continuity of probability comes to mind.) But this is none other than

$$P\left( \left| \overline{Y}_n - \mu \right| > \epsilon \quad i.o. \right) = 0$$

If $$Y$$ approximates a non-discrete r.v. $$X$$, then their sample means can be made as close as you like by refining the partition. Therefore

$$P\left( \left| \overline{X}_n - \mu \right| > \epsilon + \Delta \quad i.o. \right) = 0 \quad \forall \Delta > 0$$

So $$P\left( \left| \overline{X}_n - \mu \right| > \epsilon \quad i.o. \right) = 0$$, or $$\overline{X_n} - \mu \xrightarrow{a.s.} 0$$

Where's the wrong step? Thanks! -JP