Suppose that the random variables $X_1,\ldots, X_n$ are independent and identically distributed with mean $\mu$ and variance $\sigma^2$. Let $\bar X = (X_1 + \cdots + X_n)/n$ be the sample mean and $S = ((X_1 - \bar X)^2 + \cdots + (X_n - \bar X)^2)/(n-1)$ be the sample variance.

If the random variables $X_i$ are normally distributed, then $\sqrt{n}(\bar X - \mu)/\sigma$ is normally distributed and $\sqrt{n}(\bar X - \mu)/S$ has a $t$-distribution. Moreover, even if the random variables $X_i$ are not normally distributed, then $\sqrt{n}(\bar X - \mu)/\sigma$ is approximately normally distributed, according to the central limit theorem.

Question: In this case (where the random variables $X_i$ are not normally distributed), does the central limit theorem also imply that $\sqrt{n}(\bar X - \mu)/S$ has approximately a $t$-distribution?

This question arises in the context of constructing confidence intervals for the mean of a random variable, in the case where the variance is unknown.

The following passage appears in Ross's book Introduction to Probability and Statistics for Engineers and Scientists:

Our derivations of the $100(1-\alpha)$ percent confidence intervals for the population mean $\mu$ have assumed that the population distribution is normal. However, even when this is not the case, if the sample size is reasonably large then the intervals obtained will still be approximate $100(1-\alpha)$ percent confidence intervals for $\mu$. This is true because, by the central limit theorem, $\sqrt{n}(\bar X - \mu)/\sigma$ will have approximately a normal distribution, and $\sqrt{n}(\bar X - \mu)/S$ will have approximately a $t$-distribution.

I'm trying to understand the final statement about $\sqrt{n}(\bar X - \mu)/S$ having approximately a $t$-distribution.

  • $\begingroup$ The central limit theorem states, under appropriate conditions, that $\sqrt{n}(\overline{X}-\mu)/S \overset{\mathcal{D}}{\rightarrow} N(0, 1)$. It has nothing to do with the $t$-distribution. For finite $n$, the exact distribution would be $t$-distributed if the $X_i$ are themselves normal. When the $X_i$ are not normal, anything goes in terms of the finite-sample properties. $\endgroup$ – Tom Chen Feb 5 at 23:12
  • $\begingroup$ @TomChen Thanks. I added a quote from a book I'm reading -- I don't see the justification for the final statement about $\sqrt{n}(\bar X - \mu)/S$ having approximately a $t$-distribution. $\endgroup$ – eternalGoldenBraid Feb 5 at 23:13
  • $\begingroup$ As the degrees of freedom tends to $\infty$, $S\to\sigma$ in probability and the $t$-distirbution approaches $N(0,1)$. $\endgroup$ – kimchi lover Feb 5 at 23:14
  • $\begingroup$ @eternalGoldenBraid Sorry, my comment came almost the same time as your edit, adding in the passage. Presumably, for finite $n$, $S$ does suffer from being variable in itself, and that should be accounted for. Heuristically, it would make sense to use a distribution with heavier tails to account for this additional variability. But, there is no reason to choose $t$ other than our familiarity with it. $\endgroup$ – Tom Chen Feb 5 at 23:16
  • 2
    $\begingroup$ The argument goes that in most realistic cases, $S^2$ may sometimes be small by chance and so $\frac1S$ may be large making $\sqrt{n}(\bar X - \mu)/S$ also large in magnitude. The $t$-distribution handles this issue exactly when the underlying population is normally distributed with its heavier tails and its variance above $1$, and can often be a better approximation than the normal distribution in other cases $\endgroup$ – Henry Feb 6 at 0:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.