Here is my understanding of the central limit theorem: The arithmetic sum/mean of a sufficiently large number of iterates of independent random variables approaches a normal distribution.

These variables should be independent.

I can visualize with a few examples how as a chance process (analogous to drawing from a box) is repeated, the probability histogram centered at the expected sum with standard error as a SD starts to resemble a normal curve, whether or not the contents of the box originally do resemble that.

However, why is this so? I don't even know where to start.

  • 3
    $\begingroup$ The proof I have seen uses characteristic functions. $\endgroup$
    – Ian
    Aug 22 '16 at 3:53
  • $\begingroup$ There are several ways of answering this question. One is by looking at moment generating or (better) characteristic functions and doing a Taylor expansion. Another is to view the normal as an attractive fixed point under iterated convolution in the space of finite variance probabilities densities. Yet another is to view it as the density with maximum entropy given a fixed mean and variance. An excellent reference is Fischer's A history of the central limit theorem. $\endgroup$ Aug 22 '16 at 4:13
  • $\begingroup$ Terry Tao has notes on the central limit theorem, including the proof via characteristic functions: terrytao.wordpress.com/2010/01/05/… $\endgroup$ Aug 22 '16 at 4:14
  • $\begingroup$ The variables should be identically distributed, not just independent. For the classical central limit theorem, they should have finite $\sigma^2$ (and thus also a finite mean). You can relax that condition somewhat, but badly behaved distributions don't become normal on averaging; they do, however, follow a different central limit theorem in some other conditions. The proof via characteristic functions is easier and the standard method, but I find the fixed point argument @symplectomorphic mentioned above more compelling. $\endgroup$
    – anomaly
    Aug 22 '16 at 4:21
  • $\begingroup$ @anomaly: I find it more compelling too, and asked about it on CV here. The paper by Hamedani views it as a contraction argument. $\endgroup$ Aug 22 '16 at 4:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.