what the central limit theorem says

Asked what the central limit theorem says, a student replies, "as you take larger and larger samples from a population, the histogram of the sample values looks more and more Normal". Is the student right? Explain your answer.

My answer is no, the student is wrong. My explanation is the histogram of the sample values will look like the population distribution, whatever it might happen to be. The central limit theorem says that the histogram of sample means (from many large samples) will look more and more Normal.

• You are certainly absolutely right about the histogram of sample values. Mar 1, 2014 at 5:12
• am I right about the following part: The central limit theorem says that the histogram of sample means (from many large samples) will look more and more Normal? Mar 1, 2014 at 5:18
• The CLT of course does not hold for all distributions. And "look more and more normal" is vague, as is '"many large samples." The answer by Batman (scaling) is much more precise. Mar 1, 2014 at 5:24
• What distribution does CLT apply and what is a better way to phase look more and more normal? Mar 1, 2014 at 5:28
• CLT applies to any distribution whose variance exists, and to many more. For statement, I guess I am being fussy, but I would state the result exactly (see e.g. the Wikipedia article on CLT). Mar 1, 2014 at 5:32

Maybe this helps:

Take an appropriate random variable (finite second moment). Let's say that $\frac{S_n}{n}$ is the empirical mean of the random variable, and $\mu$ the theoretical mean. In this setting, $$\displaystyle \frac{S_n}{n} - \mu$$ is the deviation of the empirical mean from the theoretical one.

What the CLT says is: with an appropriate scaling, the deviations are normally distributed, i.e.

$$\mathbb P\left( \frac{\sqrt{n}}{\sigma}\left(\frac{S_n}{n} - \mu\right) \leq x \right) \to \phi(x) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^x \exp(-\frac{y^2}{2}) dy.$$

Quoting Frank den Hollander (Large Deviations, AMS): "CLT quantifies the probability that $S_n$ differs from $\mu n$ by an amount of order $\sqrt{n}$. Deviations of this size are called "normal". [...] [Deviations of size $n$] are called "large"."

An equivalent formulation of the result above is: $$\frac{S_n}{n} \sim \mathcal N(\mu, \frac{\sigma^2}{n}),$$ so, I would say that you are right.

• when I think more about it, I change my wording a little bit: the histogram of the sample values will look like the standard normal distribution, is this the same thing as I write above before or is this right also? Mar 1, 2014 at 17:14
• The histogram of the means (each mean from a certain sample) will look like a normal with mean $\mu$ and variance $\sigma^2/n$, more and more as the size of the samples goes to infinity. Mar 1, 2014 at 17:22
• As you said before, the (frequency) histogram of the values (of a specific sample) will always reflect the probability distribution of the random variable. Mar 1, 2014 at 17:23
• I am wondering standard normal distribution is not the same as probability distribution right Mar 1, 2014 at 17:33
• put everything together: the histogram of the sample values will look like the population distribution, whatever that distribution might look like as the sample size increases. The CLT says the sample mean follows a normal distribution with mean u and variance $\sigma^2 /n$ as the sample size goes to infinity. But CLT fails to population that has fat tails such as Cauchy Distribtion. Is this right? If so, do you think I could add a little more? Mar 8, 2014 at 11:26

You're not necessarily right about the histograms. Take i.i.d. samples of a Cauchy distribution - what happens?

You need at least square integrable random variables for versions CLT to apply.

Central limit theorems roughly state that if you scale the sample mean's deviations from the true mean by an appropriate factor ($\sqrt{n}$ where $n$ is the sample size in the classical CLT), you get convergence in distribution to a normal distribution.

• "My explanation is the histogram of the sample values will look like the population distribution, whatever it might happen to be"...is right? Mar 1, 2014 at 5:23
• The sample value part is fine (you can justify that by the Glivenko-Cantelli theorem). The histogram of the sample means on the other hand is where the problem can occur. Mar 1, 2014 at 5:27
• when I think more about it, I change my wording a little bit: the histogram of the sample values will look like the standard normal distribution, is this the same thing as I write above before or is this right also? Mar 1, 2014 at 17:15
• So my original claim should work better listed here "the histogram of the sample values will look like the population distribution, whatever it might happen to be" since this applies to all distribution, right? Mar 1, 2014 at 18:01
• put everything together: the histogram of the sample values will look like the population distribution, whatever that distribution might look like as the sample size increases. The CLT says the sample mean follows a normal distribution with mean u and variance $\sigma^2 /n$ as the sample size goes to infinity. But CLT fails to population that has fat tails such as Cauchy Distribtion. Is this right? If so, do you think I could add a little more? Mar 8, 2014 at 11:27