Central limit theorem confusion If a bunch of random variables $X_i$ are independently and identically distributed with an exponential distribution, their sum apparently follows a Gamma distribution.
But doesn't the central limit theorem imply that (for $X_i$ of any distribution with mean zero and variance $\sigma^2$), the sum $\sum_{i=1}^n X_i$ will become approximately normally distributed $~N(0,n\sigma^2)$ for large enough $n$ ?
Obviously I am missing something basic, but what's going on? How can the sum of i.i.d. exponential random variables have a Gamma distribution, but also be converging to normality?
 A: There are several confusions here (I was also very confused when I started learning about that topic :-).)


*

*Exponential random variables have a non zero mean (and are positive). The quantity you should be looking at, which asymptotically converges in distribution to a normal variable is $$\sqrt{n} \left( \frac{\sum_{i = 1}^n X_i}{n} - \mu \right)$$ The $\sqrt{n}$ was essential here, otherwise the distribution of the average will converge to a point mass at $\mu$. That quantity will converge to $N(0,\sigma^2)$. Both $\mu$ and $\sigma$ will be determined by the parameter of the exponential distribution.

*The central limit theorem is asymptotic. The quantity $\sqrt{n} \left( \frac{\sum_{i = 1}^n X_i}{n} - \mu \right)$ will have a distribution. Let's call it $F_n$. (it is essential to remember that it depends on $n$). $F_n$ in general is not a normal distribution $N(0,\sigma^2)$. The central limit theorem tells us that that distribution gets in a certain sense closer and closer to $N(0,\sigma^2)$ as $n \to \infty$.

A: Good question! The Gamma distribution itself converges to a Normal distribution--exactly the Normal distribution that the Central Limit Theorem says the sum of the Exponential random variables converges to.
Of course, for any finite $n$ the distribution of the sum of Exponentials (the Gamma distribution) is not a normal distribution, since it's bounded below by zero, but as $n$ gets larger it gets closer and closer to a Normal distribution.
Here's a page that discusses the approximation, with some graphs that show how the Gamma distribution looks like a Normal when the first parameter (= the number of Exponential random variables you are summing up) gets large.
