If $X$~$N(1,0.1)$, and $Y=X^n$ where $n$ is very large (e.g. 200).

Due to the asymetric distribution, $E(Y)$ will be skewed very far towards the extreme upper end of the distribution, dominated by values of $X$ that do not even appear in most samples.

The Central Limit Theorem states that the distribution of sample means, for a large enough sample size and number of samples, follows a normal distribution with mean equal to $E(Y)$. However, given in this case the population mean is dominated by values that will almost never in practice feature in the sample, it will be extremely rare for the sample mean to be as large as the population mean.

Therefore, the distribution of sample means is not normally distributed around the population mean (which I have checked empirically).

How does this fit in with the laws of the Central Limit Theorem?



1 Answer 1


I think the crucial factor here is that the Central Limit Theorem's statement depends on the variance of the distribution from which you sample.

The expectations of powers of a normal random variable are known. Instead of thinking of $X\sim N(1, 0.1)$, we can think of $Z=X-1$ because then $Z\sim N(0, 0.1)$ which makes the moments simpler to compute. We then have $Y=X^n = (Z+1)^n$. Now: $$ E(Y)=E[(Z+1)^n] =\sum_{k=0}^n {n \choose k}E(Z^k) =\sum_{k=0}^n \delta(k\mod2) {n \choose k} \sigma^k(k-1)!!, $$ which is non-negative and finite but eventually grows very rapidly with $n$ because the double factorial grows faster than $\sigma^k$ decays. We can compute the variance of $Y$, $$ V(Y) = E(Y^2)-E(Y)^2, $$ which is also non-negative and finite but will grow very fast with $n$ as well (the positive term overwhelms the negative, again because of the double factorial).

The Central Limit Theorem states that for a random sample $A_1, A_2, \dots, A_p$, where $A_i$ are independent, identically distributed with mean $\mu_A$ and variance $\sigma_A^2$, a normalised error of the sample mean as an estimator of the mean converges in distribution to a normal variable; precisely: $$ \sqrt{p}\,(S_p-\mu_A) \xrightarrow[]{d} N(0,\sigma_A^2), $$ where $p$ is the number of samples. Not very formally, this means that the error, the random variable $(S_p-\mu_A)$, is approximately distributed like $N(0, \frac{\sigma_A^2}{p})$. As the number of samples $p$ grows, the distribution of the error is both closer to a normal distribution and has a lower variance.

However, if the variance $\sigma_A^2$ is much larger than $p$, the variange of the error, approximately $\frac{\sigma_A^2}{p}$, can still have a large variance, i.e. $S_p$ could be very different from $\mu_A$ with large probability. I think this is exactly what is happening here, because the variance of $Y$ is enormous for large $n$.

  • $\begingroup$ Thanks for the answer. The part that does not make sense to me still is where you have stated that the error is approximately distributed with mean zero, and potentially very large variance. In practice this would lead to sample means very far from the population mean, but with no pattern of being consistently above or below it. However in this case I find every sample mean to be far below the population mean, due to the population mean being so skewed by extreme values that never appear in sample, suggesting the mean error is in fact not 0. $\endgroup$
    – Joe Bloggs
    Mar 4, 2019 at 17:00
  • $\begingroup$ Well, CLT tells me that the distribution of the normalised error converges to a normal distribution, but it does not tell me how fast. In practice the convergence could be really slow, so constantly obtaining sample means below the population mean is not a sign that the theorem does not apply, simply that you need many more samples to adequately capture the distribution. You say it yourself, the extreme values are very hard to capture in practice, so your samples are going to be skewed; taking $n$ large will probably require a huge $p$ (huge like a factorial) to capture the correct behaviour. $\endgroup$
    – R_B
    Mar 4, 2019 at 17:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .