what the central limit theorem says Asked what the central limit theorem says, a student replies, "as you take larger
and larger samples from a population, the histogram of the sample values looks more and
more Normal". Is the student right? Explain your answer.
My answer is no, the student is wrong. My explanation is the histogram of the sample values will look like the population distribution, whatever it might happen to be. The central limit theorem says that the histogram of sample means (from many large samples) will look more and more Normal.
Am I right about it? It is that simple? Is there anything more I can say about this?
 A: Maybe this helps:
Take an appropriate random variable (finite second moment). Let's say that $\frac{S_n}{n}$ is the empirical mean of the random variable, and $\mu$ the theoretical mean. In this setting, $$\displaystyle \frac{S_n}{n} - \mu$$ is the deviation of the empirical mean from the theoretical one.
What the CLT says is: with an appropriate scaling, the deviations are normally distributed, i.e.
$$\mathbb P\left( \frac{\sqrt{n}}{\sigma}\left(\frac{S_n}{n} - \mu\right) \leq x \right) \to \phi(x) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^x \exp(-\frac{y^2}{2}) dy.$$
Quoting Frank den Hollander (Large Deviations, AMS): "CLT quantifies the probability that $S_n$ differs from $\mu n$ by an amount of order $\sqrt{n}$. Deviations of this size are called "normal". [...] [Deviations of size $n$] are called "large"."
An equivalent formulation of the result above is:
$$\frac{S_n}{n} \sim \mathcal N(\mu, \frac{\sigma^2}{n}),$$
so, I would say that you are right.
A: You're not necessarily right about the histograms. Take i.i.d. samples of a Cauchy distribution - what happens? 
You need at least square integrable random variables for versions CLT to apply. 
Central limit theorems roughly state that if you scale the sample mean's deviations from the true mean by an appropriate factor ($\sqrt{n}$ where $n$ is the sample size in the classical CLT), you get convergence in distribution to a normal distribution. 
