# Some case when the central limit theorem fails

If I understand correctly, for various versions of the central limit theorems (CLT), when applying to a sequence of random variables, each random variable is required to have finite mean and finite variance, plus some other conditions depending on the version of the CLT.

Months ago, I heard of something, probably about some case when the classical CLT fails, which I haven't been able to understand. I am not sure if my following description is correct, but that is perhaps the best I can recall:

if one random variable in the sequence dominates (in some sense, such as in terms of magnitude?) the other random variables, then the central limit theorem doesn't hold.

I was wondering if someone is able to figure out what the quote is trying to say?

Thanks and regards!

PS: A paper named Asymptotic Distribution Theory for the Kalman Filter State Estimator was mentioned regarding the above quote. I don't quite understand the paper, so cannot figure out how it helps to clarify the quote. But I guess Section "3.2 Remarks on Theorems and Corollaries" on page "1999" might be related.

-

There are various ways in which the CLT can "fail", depending on which hypotheses are violated. Here's one. Suppose $X_k$ are independent random variables with $E[X_k] = \mu_k$ and variances $\sigma_k^2$, and let $s_n^2 = \sum_{k=1}^n \sigma_k^2$ and $S_n = \sum_{k=1}^n (X_k - \mu_k)$. Suppose also that $\max_{k \le n} \sigma_k/s_n \to 0$ as $n \to \infty$ (so in that sense no $X_k$ is "dominant" in $S_n$). Then Lindeberg's condition is both necessary and sufficient for $S_n/s_n$ to converge in distribution to ${\mathscr N}(0,1)$.

EDIT: Here's a nice example where the Central Limit Theorem fails. Let $X_n$ be independent with $P(X_n = 2^n) = P(X_n = -2^n) = 2^{-2n-1}$, $P(X_n = 0) = 1 - 2^{-2n}$. Thus $E[X_n] = 0$ and $\sigma_n = 1$. But $$P(S_n = 0) \ge P(X_j = 0 \text{ for all }j) > 1 - \sum_{j=1}^\infty 2^{-2j} = 2/3$$

-
Thanks! I wonder what showing $P(S_n=0) > 2/3$ is meant for? It is not to show that Lindeberg's condition fails to be satisfied, is it? – Tim Jul 1 '12 at 15:26
It shows directly that $S_n/s_n$ does not go to ${\cal N}(0,1)$ in distribution. A normal distribution has $P(Z=0) = 0$. – Robert Israel Jul 1 '12 at 18:37
Thanks! A related question about Lindeberg CLT . Now back to your example. (1) I know that Lindeberg condition is not a necessary condition, but it is not satisfied by your example (take $\epsilon=1$ in Lindeberg condition for example), so it is not surprising that $S_n/s_n$ does not converge to $N(0,1)$. (2) Another issue. The example does satisfy another condition $\lim_n S_n/n < \infty$ which is meant to replace Lindeberg condition in Greene's Econometrics. I wonder where it is gone wrong? – Tim Jul 3 '12 at 20:58

A related heuristics is as follows. Consider an i.i.d. sequence $(X_n)_{n\geqslant1}$ with $\mathrm P(X_n\gt0)\ne0$. For every $n\geqslant2$, call $T_n$ the maximum of the set $\{X_k\mid 1\leqslant k\leqslant n\}$ and $U_n$ its second maximal value. (In case of a tie at the maximum, let $U_n=T_n$.)

If $\mathrm P(T_n\leqslant xU_n)\to0$ for every $x\geqslant1$ when $n\to\infty$, that is, if $T_n/U_n\to\infty$ in probability, then the central limit theorem does not hold for the sequence $(X_n)_{n\geqslant1}$.

-
Thanks! In the example, I wonder which version of CLT doesn't hold? In that case, does $X_n$ have finite mean or finite variance? I guess not because otherwise the classical CLT still holds for $(X_n)$? – Tim Jul 1 '12 at 7:26

This doesn't directly answer your question as to what the quote is saying but a good example to keep in mind as an example when the CLT fails.

If $\{X_k\}_{k=1}^{n}$ are independent and identically distributed standard Cauchy random variables, then the sample mean $\left(\displaystyle \sum_{k=1}^n X_k \right)/n$ has the same standard Cauchy distribution. This highlights the importance of finite variance in the CLT.

-

A simple example is given by what in Signal Processing is called an AR 1 process (autoregressive order 1), say:

$$x[n] = a \; x[n-1] + e[n] \hspace{1cm } |a|<1$$

$e[n]$ is a sequence of iid variables (a stationary white noise process), with arbitrary distribution, zero mean and finite variance. Then, the question arise whether $x[n]$ is asymptotically gaussian. Because, letting $x[0]=0$ we have

$$\begin{eqnarray} x[1]&=&e[1]\\ x[2]&=&a \,e[1] + e[2]\\ x[3]&=& a^2 \, e[1] + a \, e[2] + e[3]\\ x[n]&=& a^{n-1} e[1] + a^{n-2}e[2] + \cdots + a \, e[n-1] + e[n] \end{eqnarray}$$

Then, $x[n]$ is the sum of $n$ independent (not iid) variables ($a^{n-i} e[i]$) with finite variance. For large $n$, can we then apply the CLT and conclude that $x[n]$ will asymptotically tend to a gaussian, regardless of the distribution of $e[n]$? The answer is no, because the conditions do not apply here.

And indeed, it can be checked that if, for example, $e[n]$ is uniform in $[-1,1]$, then $|x[n]|$ is always bounded to $1/(1-a)$ (geometric series), and hence it cannot converge to a gaussian.

-

I remember this paper on floating point arithmetic talking about something like that. http://www.cs.berkeley.edu/~wkahan/improber.pdf

It's about how trying to use normally distributed errors in floating point calculations to increase statistical confidence in the result fails because the error is often dominated by a single term, and has some nice graphs and examples. It really goes to town on a guy named Vignes who "rediscovered" the technique decades after it was known not to work, patented it, and got an award for inventing it from the French Academy of Sciences.

The paper stuck in my mind because I read it in 2006 and it gave a small example of the phenomenon in insurance policy: how the assumption of normality can lead to financial models that drastically underestimate the risk of catastrophic failure.

-