Some case when the central limit theorem fails

If I understand correctly, for various versions of the central limit theorems (CLT), when applying to a sequence of random variables, each random variable is required to have finite mean and finite variance, plus some other conditions depending on the version of the CLT.

Months ago, I heard of something, probably about some case when the classical CLT fails, which I haven't been able to understand. I am not sure if my following description is correct, but that is perhaps the best I can recall:

if one random variable in the sequence dominates (in some sense, such as in terms of magnitude?) the other random variables, then the central limit theorem doesn't hold.

I was wondering if someone is able to figure out what the quote is trying to say?

Thanks and regards!

PS: A paper named Asymptotic Distribution Theory for the Kalman Filter State Estimator was mentioned regarding the above quote. I don't quite understand the paper, so cannot figure out how it helps to clarify the quote. But I guess Section "3.2 Remarks on Theorems and Corollaries" on page "1999" might be related.

There are various ways in which the CLT can "fail", depending on which hypotheses are violated. Here's one. Suppose $X_k$ are independent random variables with $E[X_k] = \mu_k$ and variances $\sigma_k^2$, and let $s_n^2 = \sum_{k=1}^n \sigma_k^2$ and $S_n = \sum_{k=1}^n (X_k - \mu_k)$. Suppose also that $\max_{k \le n} \sigma_k/s_n \to 0$ as $n \to \infty$ (so in that sense no $X_k$ is "dominant" in $S_n$). Then Lindeberg's condition is both necessary and sufficient for $S_n/s_n$ to converge in distribution to ${\mathscr N}(0,1)$.

EDIT: Here's a nice example where the Central Limit Theorem fails. Let $X_n$ be independent with $P(X_n = 2^n) = P(X_n = -2^n) = 2^{-2n-1}$, $P(X_n = 0) = 1 - 2^{-2n}$. Thus $E[X_n] = 0$ and $\sigma_n = 1$. But $$P(S_n = 0) \ge P(X_j = 0 \text{ for all }j) > 1 - \sum_{j=1}^\infty 2^{-2j} = 2/3$$

• Thanks! I wonder what showing $P(S_n=0) > 2/3$ is meant for? It is not to show that Lindeberg's condition fails to be satisfied, is it?
– Tim
Jul 1 '12 at 15:26
• It shows directly that $S_n/s_n$ does not go to ${\cal N}(0,1)$ in distribution. A normal distribution has $P(Z=0) = 0$. Jul 1 '12 at 18:37
• Thanks! A related question about Lindeberg CLT . Now back to your example. (1) I know that Lindeberg condition is not a necessary condition, but it is not satisfied by your example (take $\epsilon=1$ in Lindeberg condition for example), so it is not surprising that $S_n/s_n$ does not converge to $N(0,1)$. (2) Another issue. The example does satisfy another condition $\lim_n S_n/n < \infty$ which is meant to replace Lindeberg condition in Greene's Econometrics. I wonder where it is gone wrong?
– Tim
Jul 3 '12 at 20:58

This doesn't directly answer your question as to what the quote is saying but a good example to keep in mind as an example when the CLT fails.

If $\{X_k\}_{k=1}^{n}$ are independent and identically distributed standard Cauchy random variables, then the sample mean $\left(\displaystyle \sum_{k=1}^n X_k \right)/n$ has the same standard Cauchy distribution. This highlights the importance of finite variance in the CLT.

A related heuristics is as follows. Consider an i.i.d. sequence $(X_n)_{n\geqslant1}$ with $\mathrm P(X_n\gt0)\ne0$. For every $n\geqslant2$, call $T_n$ the maximum of the set $\{X_k\mid 1\leqslant k\leqslant n\}$ and $U_n$ its second maximal value. (In case of a tie at the maximum, let $U_n=T_n$.)

If $\mathrm P(T_n\leqslant xU_n)\to0$ for every $x\geqslant1$ when $n\to\infty$, that is, if $T_n/U_n\to\infty$ in probability, then the central limit theorem does not hold for the sequence $(X_n)_{n\geqslant1}$.

• Thanks! In the example, I wonder which version of CLT doesn't hold? In that case, does $X_n$ have finite mean or finite variance? I guess not because otherwise the classical CLT still holds for $(X_n)$?
– Tim
Jul 1 '12 at 7:26

A simple example is given by what in Signal Processing is called an AR 1 process (autoregressive order 1), say:

$$x_n = a \; x_{n-1} + e_n \hspace{1cm } |a|<1$$

where $x_0=0$ and $e_n$ is a sequence of iid variables (a stationary white noise process), with arbitrary distribution, zero mean and finite variance. Then, the question arise whether $x_n$ is asymptotically gaussian. Because we have:

$$\begin{eqnarray} x_1&=&e_1\\ x_2&=&a \,e_1 + e_2\\ x_3&=& a^2 \, e_1 + a \, e_2 + e_3\\ x_n&=& a^{n-1} e_1 + a^{n-2}e_2 + \cdots + a \, e_{n-1} + e_n = \sum_{i=1}^n a^{n-i} e_i \end{eqnarray}$$

Therefore, $x_n$ is the sum of $n$ independent (not iid) variables ($z_i=a^{n-i} e_i$). Now, can we apply the CLT and conclude that, regardless of the distribution of $e_n$ (as long as they have a fixed finite variance), $x_n$ tends to a gaussian for large $n$? The answer is no, because the conditions do not apply here.

Indeed, if, for example, $e_n$ is uniform in $[-1,1]$, then $|x_n|$ is always bounded below $1/(1-a)$ (geometric series), and hence it cannot converge to a gaussian.

I remember this paper on floating point arithmetic talking about something like that. http://www.cs.berkeley.edu/~wkahan/improber.pdf

It's about how trying to use normally distributed errors in floating point calculations to increase statistical confidence in the result fails because the error is often dominated by a single term, and has some nice graphs and examples. It really goes to town on a guy named Vignes who "rediscovered" the technique decades after it was known not to work, patented it, and got an award for inventing it from the French Academy of Sciences.

The paper stuck in my mind because I read it in 2006 and it gave a small example of the phenomenon in insurance policy: how the assumption of normality can lead to financial models that drastically underestimate the risk of catastrophic failure.

The main idea of the quote is that the CLT arises from many offsetting terms that mostly subtract each other out when you add them together. Having a few terms dominate throws everything off.

An important example is the $\alpha$-stable distributions. Their densities can’t usually be written explicitly, but they have the property that they decay like $|x|^{-1-\alpha}$ where $0<\alpha<2$. This means that they don’t have a variance (and can’t satisfy the CLT), and when $\alpha\le1$ they don’t even have a mean. They do satisfy a CLT like theorem though: if $X_i$ are i.i.d. $\alpha$-stable, then $n^{-1/\alpha}\sum X_i$ is $\alpha$-stable. Notice that $\alpha=2$ corresponds to the regular CLT.

This brings us to your quote. We’ll fix a constant $c>0$, and examine $S=n^{-1/\alpha}\sum X_i 1_{X_i<cn^{1/\alpha}}$ and $B=n^{-1/\alpha}\sum X_i 1_{X_i>cn^{1/\alpha}}$. In other words, the CLT-like sum, but only using the variables that are Smaller or Bigger than a growing cutoff. In the $\alpha=2$ CLT case, $B\to0$ while $S$ converges to the normal random variable. In the $\alpha<2$ case, however, $S$ is bounded while $B$ converges to the $\alpha$ stable random variable.

This means that for the variable $n^{-1/\alpha}\sum X_i$ in the $\alpha=2$ case, the large values are negligible; whereas in the $\alpha<2$ case, the sum is mostly made up of a few large values that have dominated the rest.