I was curious...

There are several problems in a intro to probability textbook that ask you to prove that:

  1. If IID sequence is of Geometric RVs, then the sum of these random variables is a new random variable with a negative binomial distribution

  2. if IID sequence is of Exponential RVs, then the sum of these random variables is a new random variable that has a gamma distribution

  3. if IID sequence is of Poisson RVs, then the sum of these random variables is a new random variable that is also a poisson RV.

  4. Summing squared IID N(0,1) RV, gives you a RV with a chi-squared distribution...


What I don't get, is how this fits in with Central Limit Theorem?

CLT: Summing IID RV approaches a Gaussian distribution as sample size approaches infinity.

Isn't this a contradiction to the other proofs of adding other types of RVs and NOT getting a Gaussian distribution? what gives?


5 Answers 5


The CLT holds only for (1) $n\to \infty$, and (2) appropriate scaling, while the exact distributions that you stated hold only for (1) finite $n$ and (2) without scaling. Any other variations are mere approximations and not exact results.


There is no contradiction. What the CLT is telling you is that the 'output' distributions that you've named: (1) the negative binomial, (2) the gamma, (3) the Poisson, (4) the chi-squared, and of course (5) the binomial distribution, all have normal approximations, when the distributions in question arise as the sum of $n$ iid random variables, and after suitable scaling. Moreover, the CLT asserts that the approximation improves with $n$. You should be able to find questions on Math.SE asking about the normal approximation to each of these distributions.


What you wrote is not a definition of CLT. Instead, it is $$ \frac{S_n - n \mu}{\sigma \sqrt{n}} \to_n N(0,1) $$ assuming each $X_i$ is integrable and has a finite second moment, and all rvs are iid. This doesn't contradict any of the examples you gave, because they are specific to the distribution of those rvs; CLT applies to all rvs that fulfill the requirements of CLT.

  • $\begingroup$ if you remove the streching and shifting of $Z = \frac{S_n -n\mu}{\sigma\sqrt{n}} $ then its $\lim \limits_{n\to \infty}S_n =\sum \limits_{i} X_i ~~~\to ~~~~N(\mu, \sigma^2)$ which is just summing of random variables... and contradicts... $\endgroup$
    – pico
    Jul 26, 2020 at 23:14
  • $\begingroup$ Why do you keep throwing around the word "contradict" when it is plainly wrong? Everywhere that gives a precise statement of CLT will say clearly that it concerns convergence in distribution, which means the cumulative density function of the appropriately translated and scaled sum converges pointwise to that of the standard normal. You cannot anyhow 'remove' this or that and still claim to have that! Otherwise it is like claiming that as $n→∞$ we have $1+\frac1{n} → 1$ and so $n+1 → n$ as well. $\endgroup$
    – user21820
    Jul 27, 2020 at 7:55

CLT says something approaches a Gaussian distribution, not that it is a Gaussian distribution.

Suppose $X\sim\operatorname{Poisson}(\lambda).$ Then the distribution of $\dfrac{X-\lambda}{\sqrt\lambda}$ approaches $\operatorname N(0,1)$ as $\lambda\to\infty.$

Suppose $X\sim\operatorname{Gamma}$ of the form $\displaystyle \frac 1 {\Gamma(\alpha)} \left( \frac x \mu \right)^{\alpha-1} e^{-x/\mu} \left( \frac{dx}\mu \right)$ for $x>0.$ Then the expected value is $\alpha\mu$ and the variance is $\alpha\mu^2,$ so $\dfrac{X-\alpha\mu}{\mu\sqrt\alpha}$ approaches $\operatorname N(0,1)$ as $\alpha\to\infty.$ A gamma distribution with a large value of $\alpha$ is approximately a normal distribution with the same expected value and the same variance.


Let's take the case of the Poisson RV's as an example of what is going on.

The distribution of a Poisson RV is $$ P_\lambda(k)=\frac{\lambda^ke^{-\lambda}}{k!}\tag1 $$ where $\lambda$ is the expected value of the RV; it is also the variance of the RV. Summing $n$ RVs gives a distribution of $$ P_{n\lambda}(k)=\frac{(n\lambda)^ke^{-n\lambda}}{k!}\tag2 $$ To make the mean $0$, we translate by $n\lambda$: $$ P_{n\lambda}(k+n\lambda)=\frac{(n\lambda)^{k+n\lambda}e^{-n\lambda}}{(k+n\lambda)!}\tag3 $$ Note that as with any convolution of random variables, this produces a thinned out distribution; that is, the distribution gets spread out over a larger range and the probability for any given range is decreased commensurately. To counter the effect of this thinning, we scale back by $\sqrt{n}$ to get a distribution with mean $0$ and variance $\lambda$: $$ \begin{align} \sqrt{n}P_{n\lambda}(\sqrt{n}k+n\lambda) &=\sqrt{n}\frac{(n\lambda)^{\sqrt{n}k+n\lambda}e^{-n\lambda}}{\left(\sqrt{n}k+n\lambda\right)!}\tag4\\[6pt] &\sim\frac{\sqrt{n}}{\sqrt{2\pi}}\frac{(n\lambda)^{\sqrt{n}k+n\lambda}e^{-n\lambda}e^{\sqrt{n}k+n\lambda}}{\left(\sqrt{n}k+n\lambda\right)^{\sqrt{n}k+n\lambda+1/2}}\tag5\\[3pt] &=\frac1{\sqrt{2\pi\lambda}}\frac{(n\lambda)^{\sqrt{n}k+n\lambda+1/2}e^{\sqrt{n}k}}{\left(\sqrt{n}k+n\lambda\right)^{\sqrt{n}k+n\lambda+1/2}}\tag6\\[3pt] &=\frac1{\sqrt{2\pi\lambda}}\frac{e^{\sqrt{n}k}}{\left(\vcenter{1+\frac{k}{\sqrt{n}\lambda}}\right)^{\sqrt{n}k+n\lambda+1/2}}\tag7\\ &\sim\frac1{\sqrt{2\pi\lambda}}e^{-\frac{k^2}{2\lambda}}\tag8 \end{align} $$ Explanation:
$(4)$: substitute $\sqrt{n}k\mapsto k$ in $(3)$ and multiply by $\sqrt{n}$
$(5)$: apply Stirling's Approximation
$(6)$: cancel $e^{n\lambda}$ and $(n\lambda)^{1/2}$
$(7)$: cancel $(n\lambda)^{\sqrt{n}k+n\lambda+1/2}$
$(8)$: $\left(\vcenter{1+\frac{k}{\sqrt{n}\lambda}}\right)^{\sqrt{n}k}\sim e^{\frac{k^2}\lambda}$
$\phantom{\text{(8):}}$ $\left(\vcenter{1+\frac{k}{\sqrt{n}\lambda}}\right)^{n\lambda}\sim e^{\sqrt{n}k-\frac{k^2}{2\lambda}}$
$\phantom{\text{(8):}}$ $\left(\vcenter{1+\frac{k}{\sqrt{n}\lambda}}\right)^{1/2}\sim 1$

Formula $(8)$ is normal distribution with mean $0$ and variance $\lambda$.

Here is the Poisson distribution, $P_{n\lambda}$ for $\lambda=1$, scaled and plotted for various $n$. The points are placed so that $\sqrt{n}k+n\lambda\in\mathbb{Z}$.

The continuous curve is the Normal distribution with variance $1$.

Poisson to Normal distribution

Thus, the Poisson distribution, when scaled to counter the thinning, tends to a normal distribution.

At no point is the Poisson distribution a normal distribution. For one thing for any $n$, $P_{n\lambda}$ is a discrete distribution; the normal distribution is a continuous distribution. This is a property of limits: it is not necessary that at any point, a sequence equals its limit.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.