1
$\begingroup$

My question: Is it only the probability distribution of the sample mean statistic of a sample which is normally distributed according to the Central Limit Theorem, or, will any statistic work like for example the sample variance?


My lack of clarity comes from reading the things below.

"A sampling distribution is the probability distribution of a given random-sample-based statistic." - Wikipedia.

"Central Limit Theorem (CLT) establishes that, in some situations, when independent random variables are added, their normalized sum tends toward a normal distribution even if the original variables themselves are not normally distributed." - Wikipedia.
(Makes it seem the CLT only works for the sampling distribution of the sample mean statistic.)


"Let $\{X_1, X_2,...,X_n\}$ be a random sequence of i.i.d. random variables drawn from a distribution with expected value $\mu$ and finite variance $\sigma^2$ - i.e. a random sample size $n$.

The CLT says that ${\bf \sqrt{n}(\frac{X_1+X_2+...+X_n}{n}-\mu)\xrightarrow{d}N(0,\sigma^2)}$ (Lindeberg-Lévy CLT)." Illustrated with the following picture: enter image description here


But then when I search for "the CLT" on the internet I find pictures like these:

enter image description here

That talk about a sampling distribution becoming normal without any mention of that sampling distribution necessarily being that of a sample mean.

$\endgroup$
5
  • 2
    $\begingroup$ The picture isn't entirely correct as central limit theorem can fail as in the Cauchy distribution. $\endgroup$
    – Karl
    Feb 26, 2020 at 16:10
  • 1
    $\begingroup$ As far I recall the sample variance would have a chisquare distribution with $n-1$ degrees of freedom. But the chisquare distribution is itself the sum of squared standard normal variables so will so will look normal under the CLT for large n. $\endgroup$
    – Karl
    Feb 26, 2020 at 16:21
  • $\begingroup$ @Karl Okay, so my take away is: the sampling distribution of sample mean will be normal for sure under CLT, and, other sample statistics will in some cases be normal too but might need a larger sample size for the sampling distribution to approximate a normal distribution, but for some statistics the sampling distribution will approximate some other distribution like the Chi-square distribution. Correct? $\endgroup$ Feb 26, 2020 at 16:27
  • 1
    $\begingroup$ My understanding is that the CLT is concerned with the sum of independent random variables and states that distribution is approximately normal for large n with some exceptions. The sample variance would have a chisquare distribution. The sum of two chisquar distributions is also chisquare but as you add more chisquare then the distribution of the sum will be chisquare but looking more normal. I'm no expert and there will be more satisfactory answers both here and on cross validated. I can't give more detail as I don't want to mislead. $\endgroup$
    – Karl
    Feb 26, 2020 at 16:42
  • 1
    $\begingroup$ Final comment if I may. The CLT is concerned with summing random variables. In this light it makes sense why the biniomal distribution tends to a normal for large n as the biniomal distribution can be view itself as the sum of Bernoulli distributions. I guess I'm saying the distribution of sample means is just one application. $\endgroup$
    – Karl
    Feb 26, 2020 at 17:01

1 Answer 1

3
$\begingroup$

Yes, the CLT only works for the sample mean.

The second diagram you have given is not clear, but the sampling distribution they refer to is the distribution of the sample mean.

If you work out the standard deviation of your sample and repeat it, the values will not be the same as those taken from a normal distribution. In particular, there will be a bias, which we correct for by dividing by $n-1$ instead of $n$ when we use the sample data to estimate the standard deviation of the population.

$\endgroup$
6
  • $\begingroup$ @ Great answer! I found that very many explanations fail completely to mention or clearly emphasise that the CLT only is applicable for one specific statistic - the sample mean. $\endgroup$ Feb 26, 2020 at 16:33
  • 3
    $\begingroup$ CLT also works for the sample sum... $\endgroup$
    – nomen
    Feb 26, 2020 at 16:59
  • $\begingroup$ @tomi Excellent example with the standard deviation! $\endgroup$
    – liwuen
    Nov 16, 2022 at 11:25
  • 1
    $\begingroup$ @MSIS Look at this: stats.stackexchange.com/questions/45124/… $\endgroup$
    – tomi
    Oct 11, 2023 at 10:27
  • 2
    $\begingroup$ The sample variance (relocated and rescaled in a CLT sense) converges in distribution to a normal distribution as a result of applying the CLT to $(X-\mu)^2$ (provided that the fourth moment of the underlying distribution is finite). Its square-root also does, using the delta method. $\endgroup$
    – Henry
    Oct 28, 2023 at 12:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .