I have a proof (sketch) of the Strong Law of Large Numbers, at least the "sufficiency" half of it, that seems a little too easy. This is the version where you only assume i.i.d. random variables, and $E[X]<\infty$.

The idea is to derive it ultimately from the special case of i.i.d. Bernoulli r.v.s. That case is relatively elementary to prove: you can get it from the convergence of the binomial to the normal, for instance, or by showing that the 4th centralized moment is proportional to $n^2$ for the binomial distribution. The apply Borel-Cantelli, etc.

The first step is to approximate the random variable X's by discrete Y's (if they aren't already discrete). This can be done in such a way that $E[X] = E[Y] = \mu$. Simply partition the real numbers, and then compute the expectation of X on each set in the partition. Then when $X = x$ occurs, $x$ is in one of the partitioning sets, say $B_i$. Then we say that $Y = y$ has occurred, where $y$ is the average of $X$ on that partitioning set. In other words,

$$ Y = \sum_i E[X|X \in B_i]1_{\{X \in B_i\}} $$

That way, the discrepancy between $X$ and $Y$ becomes smaller as the partition becomes more refined. If the partitioning sets are intervals, then $|X-Y| \leq \Delta$, where $\Delta$ is the width of the largest interval in the partition. More importantly,

$$ |\overline{X}_n - \overline{Y}_n| \leq \Delta $$

Having approximated $X$ by a discrete r.v. $Y$, and argued that their sample means can be made arbitrarily close, it remains to show that the SLLN holds for discrete r.v.s with finite mean. If $Y$ takes values $y_1, y_2, y_3,...$ with probabilities $p_1, p_2, p_3,...$ then you can say

$$ Y = \sum_i y_i Z_i \quad \mbox{where} \quad Z_i = 1_{ Y^{-1}(y_i)} $$

In other words, you can look at it as a weighted sum of Bernoulli r.v.s with respective success probabilities $p_1, p_2,$, etc. For each of these Bernoulli random variables $Z_i$ (i.e. success if $Y=y_1$, failure otherwise), the SLLN for Bernoulli sequences tells us that

$$\overline{Z}_{i,n} - p_i \xrightarrow{a.s.} 0 $$

Since we are assuming that $\mu = \sum_i p_i y_i < \infty$, then for any $\delta > 0$ we can choose $I$ large enough so that $\sum_{i>I} p_i y_i < \delta$. Say we have some $\delta$ and its corresponding $I$ in mind. By linearity of almost sure convergence,

$$ \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^I p_iy_i \xrightarrow{a.s.} 0 $$

In other words, for any $\epsilon > 0$

$$ P\left( \left| \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^I p_iy_i \right| > \epsilon \quad i.o. \right) = 0 $$

By assumption, $\left| \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^I p_iy_i \right| > \left| \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^\infty p_iy_i \right| - \delta$. So, for each pair $(\delta, I)$

$$ P\left( \left| \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^\infty p_iy_i \right| > \epsilon + \delta \quad i.o. \right) \leq P\left( \left| \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^I p_iy_i \right| > \epsilon \quad i.o. \right) = 0 $$

That is, $ P\left( \left| \sum_{i=1}^I y_i\overline{Z}_{i,n} - \sum_{i=1}^\infty p_iy_i \right| > \epsilon + \delta \quad i.o. \right) = 0 $ for each $(\delta, I)$. Since $I$ gets large as $\delta$ gets small, I'm thinking this means that

$$ P\left( \left| \sum_{i=1}^\infty y_i\overline{Z}_{i,n} - \sum_{i=1}^\infty p_iy_i \right| > \epsilon \quad i.o. \right) = 0 $$

(This seems "obvious", but it's the step I'm not sure exactly how to justify if it needs it. Continuity of probability comes to mind.) But this is none other than

$$ P\left( \left| \overline{Y}_n - \mu \right| > \epsilon \quad i.o. \right) = 0 $$

If $Y$ approximates a non-discrete r.v. $X$, then their sample means can be made as close as you like by refining the partition. Therefore

$$ P\left( \left| \overline{X}_n - \mu \right| > \epsilon + \Delta \quad i.o. \right) = 0 \quad \forall \Delta > 0$$

So $P\left( \left| \overline{X}_n - \mu \right| > \epsilon \quad i.o. \right) = 0$, or $\overline{X_n} - \mu \xrightarrow{a.s.} 0$

Where's the wrong step? Thanks! -JP


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.