References for Kolmogorov's strong law of a large numbers

Question

On the Wikipedia law of large numbers site, they mention "Kolmogorov's strong law of large numbers", which works even if the random variables are not identically distributed.

Where can I find this theorem shown and proven? I know that a reference is provided on the Wikipedia site, but that book is out of availability. Are there any other references out there?

(Interestingly, Allan Gut's book "Probability: A Graduate Course", has a theorem by the name of "Kolmogorov's strong law", but in his book, the random variables have to be identically distributed. Any ideas why this is?)

Answer 1

It does not work, in general, if the summands are not iid. Both "independent" and "identically distributed" can be weakened, but you can't dispense with either of them entirely and still get the result without giving up something else.

The result that they cite as "Kolmogorov's Strong Law" is not what I always refer to as Kolmogorov's Strong Law (I suspect that what I refer to as Kolmogorov's Strong Law is the same thing that Allan Gut does). The result given on Wikipedia requires a finite second moment and that $\sum \frac{\mbox{Var} X_k}{k^2} < \infty$, but in exchange you can drop the requirement of being identically distributed. The proof of this version I think is actually pretty easy, the sketch being: because the series converges we can apply the Khintchine-Kolmogorov convergence theorem so that $\sum \frac{(X_k - \mu_k)}{k}$ converges almost surely, and the result follows after an application of Kronecker's Lemma.

Answer 2

There is a proof of the theorem requiring independence but not identical distribution in William Feller, An Introduction to Probability Theory and Its Applications vol. 1, 3rd ed. Wiley 1968, section 10.7.

I'm not sure why the Wikipedia page only cites Sen and Singer; by now I would have thought someone would have noticed that it's also in The Standard Reference. (In case the Sen and Singer reference on the Wikipedia page goes away, it's: Sen, P. K.; Singer, J. M. (1993), Large sample methods in statistics. Chapman & Hall, Inc., theorem 2.3.10.)

The proof in Feller is different from the one in Sen and Singer, by the way.

For a very different, nontraditional approach to this theorem, see the versions in Shafer and Vovk's 2001 or 2019 books.

Shafer and Vovk cite the original source as Kolmogorov, "Sur la loi forte des grands nombres", Comptes Rendus des Séances de l'Académie des Science v. 191 (1932), pp. 910-912, with an English translation on pp. 60-61 of Selected Works of A.N. Kolmogorov: vol. 2 Probability Theory and Mathematical Statistics (1992).