I am an aspiring probabilist, and I definitely know the central limit theorem. However, I am trying to understand what idea it really embodies.

I am aware that normal distribution arises as the limit even when the averages are taken over non-independent random variables. But in some sense, it seems the correlations among the $X_i$'s are weak. Is this intuition correct? In other words, when can we expect a central limit theorem?

Is there some deeper notion embodied in this limit law? It is so universal, it seems it is quite mysterious...

  • $\begingroup$ "I am aware that normal distribution arises as the limit even when the averages are taken over non-independent random variables." This is true in certain contexts, but not in general. Consider the case where $X_1 = X_2 = \dots$ and $X_1 = \pm 1$ with probability $0.5$. Then the sample average $S_n$ will be $1$ or $-1$, each with probability $0.5$; in no sense is there a normal distribution anywhere. Fundamentally, the $X_i$ terms are too correlated to one another for the CLT to kick in. (Perhaps this is kind of what you're asking about?) $\endgroup$ – Aaron Montgomery Nov 1 '18 at 16:24
  • $\begingroup$ Yes, so correlations being too high prevents the CLT from kicking in...but why is the limit always normal? Even in weak dependence? What is so special about it? $\endgroup$ – Landon Carter Nov 1 '18 at 16:40

Here is an answer to "what's so special about the normal distribution?"

The limiting distribution is normal because the family of normal distributions is closed under addition (and averaging): if $X$ and $Y$ are normal, so are $X+Y$ and $\frac{X+Y}{2}$, with the appropriately modified means and variances.

So if we started with a bunch of i.i.d. normal random variables, their average would stay normal. This tells us that if any sort of central limit theorem holds, it ought to give the normal distribution in the limit.

In general, whenever a limit theorem holds, it gives a distribution in the limit which is closed under the operation we care about.

There are other limit theorems for other cases. For example, the Poisson distribution is a distribution on $\mathbb N$ with a similar property: if $X$ and $Y$ are Poisson, so is $X+Y$. We have Le Cam's theorem and Janson's inequality as two instances of the claim that if we sum many approximately-independent Bernoulli random variables, then in cases where the mean of the sum is constant, it has an approximately Poisson distribution.

(This doesn't contradict the central limit theorem, because the Poisson distribution with mean $\lambda$ is well-approximated by the normal distribution with mean and variance $\lambda$ when $\lambda$ is large.)

For another example, the exponential distribution is closed under minimums and scaling: if $X$ and $Y$ are exponential, so are $k \cdot X$ and $\min\{X,Y\}$. We can prove various forms of a "minimum limit theorem": if $X_1,X_2, \dots, X_n$ are sufficiently independent nonnegative random variables with probability density functions that have nonzero and reasonably similar values at $0$, then then $n \cdot \min\{X_1, X_2, \dots, X_n\}$ should be approximately exponential.


I consider an i.i.d case., i.e. there is a sequence of i.i.d r.v.s $\{X_n\}$ with distribution $F_X$. It is known that $F_X$ belongs to the domain of attraction of some distribution $F_{\infty}$ is there exists a sequence $\{(m_n,s_n)\}$ s.t. $$ \frac{\sum_{i\le n} X_i-m_n}{s_n}\xrightarrow{d}F_{\infty}. $$

We also know that only stable distributions can be domains of attraction. In particular, $F_X$ belongs to the domain of attraction of the normal distribution if and only if $$ U(x)=\mathsf{E}X_1^2 1\{|X_1|\le x\} $$ is a slowly varying function. For more results you may look at Chapter 9 in Probability: A Graduate Course by Allan Gut.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.