Suppose that $X_1, X_2, \ldots, $ is a sequence of iid random variables with $\mathbb{E}(X_i) = \mu$ and $Var(X_i) = \sigma^2 < \infty$. Define $S_n = \sum_{i=1}^{n}X_i$.

Then as $n$ approaches infinity, we have that $\sqrt{n}(S_n - \mu)$ converges in distribution to a normal $N(0,\sigma^2)$ distribution:

$$ \sqrt{n}\left(\left(\frac{1}{n}\sum_{i=1}^n X_i\right) - \mu\right)\ \xrightarrow{d}\ N\left(0,\sigma^2\right). $$

Assuming that $\sigma>0$, convergence in distribution is taken to be that the cumulative distribution functions of $\sqrt{n}(S_n - \mu)$ converge pointwise to the cdf of the $N(0, \sigma^2)$ distribution in that for every real number $z$:

$$ \lim_{n\to\infty} \Pr\left[\sqrt{n}(S_n-\mu) \le z\right] = \Phi\left(\frac{z}{\sigma}\right) , $$

My Question: I am having a hard time understanding what is means when they say the above convergence happens with rate $\sqrt{n}$.

The notion of rate implies that it characterizes how fast something converges, but I am failing to see in the equation above where the slowness or the "fastness" is coming in. Can someone help me see this?

  • $\begingroup$ One knows that $\frac1nS_n\to\mu$ and the rate of this convergence is described by the fact that $\frac1nS_n-\mu=\frac1{\sqrt{n}}Z_n$ where $Z_n\to Z$ in distribution with $Z$ centered normal. Thus the error term $\frac1nS_n-\mu$ is roughly $\frac1{\sqrt{n}}Z$, which is clearly of order $\frac1{\sqrt{n}}$. (Note that your question confuses $S_n$ with $\frac1nS_n$ at some places, it seems.) $\endgroup$
    – Did
    May 7, 2017 at 9:00
  • $\begingroup$ My feel is like: By WLLN, $S_n$ converge to $\mu$ in probability, i.e. the difference is converge to $0$. And $\sqrt{n}$ will goes to infinity as $n$ goes to infinity. When you multiply them together, it does not explode - it converge in distribution to a bounded random variable. So if you further multiply it by a higher order of $n$, it will explode. Is it what you want? $\endgroup$
    – BGM
    May 7, 2017 at 9:04
  • $\begingroup$ @BGM - You need to divide ${S_n}-n\mu$ by $\sqrt{n}$, or multiply $\frac{S_n}{n}$ or $\frac{S_n}{n}-\mu$ by $\sqrt{n}$ $\endgroup$
    – Henry
    May 7, 2017 at 9:09

1 Answer 1


The mean of $S_n$ is $n\mu$ and the variance of $S_n$ is $n\sigma^2$ so the standard deviation of $S_n$ is $\sqrt{n}\sigma$

This has the consequence that $\dfrac{S_n}{n}$ has mean $\mu$ and variance $\dfrac{\sigma^2}{n}$ and standard deviation $\dfrac{\sigma}{\sqrt{n}}$. I suspect that this is what led to your phrase "convergence happens with rate $\sqrt{n}$"

So defining $T_n = \dfrac{S_n-n\mu}{\sqrt{n}\sigma}$, you can find $T_n$ has mean $0$ and variance $1$ and standard deviation $1$

What the central limit theorem says is that this converges in distribution to a standard normal as $n$ increases, i.e

$$\lim_{n\to\infty} \Pr\left[T_n \le t\right] = \Phi\left(t\right)$$ or $$\lim_{n\to\infty} \Pr\left[\dfrac{S_n-n\mu}{\sqrt{n}\sigma} \le t\right] = \Phi\left(t\right)$$ or $$\lim_{n\to\infty} \Pr\left[\sqrt{n}\dfrac{\frac{S_n}{n}-\mu}{\sigma} \le t\right] = \Phi\left(t\right)$$ and letting $z=t\sigma$ $$\lim_{n\to\infty} \Pr\left[\dfrac{S_n-n\mu}{\sqrt{n}} \le z\right] = \Phi\left(\frac{z}{\sigma}\right)$$ or $$\lim_{n\to\infty} \Pr\left[\sqrt{n}\left(\frac{S_n}{n}-\mu\right)\le z\right] = \Phi\left(\frac{z}{\sigma}\right)$$

which is not quite what you have written, as it is a description of the behaviour of $\frac{S_n}{n}$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.