For the Central Limit Theorem, why does the $\sqrt{n}$ term represent the convergence rate?

Suppose that $X_1, X_2, \ldots,$ is a sequence of iid random variables with $\mathbb{E}(X_i) = \mu$ and $Var(X_i) = \sigma^2 < \infty$. Define $S_n = \sum_{i=1}^{n}X_i$.

Then as $n$ approaches infinity, we have that $\sqrt{n}(S_n - \mu)$ converges in distribution to a normal $N(0,\sigma^2)$ distribution:

$$\sqrt{n}\left(\left(\frac{1}{n}\sum_{i=1}^n X_i\right) - \mu\right)\ \xrightarrow{d}\ N\left(0,\sigma^2\right).$$

Assuming that $\sigma>0$, convergence in distribution is taken to be that the cumulative distribution functions of $\sqrt{n}(S_n - \mu)$ converge pointwise to the cdf of the $N(0, \sigma^2)$ distribution in that for every real number $z$:

$$\lim_{n\to\infty} \Pr\left[\sqrt{n}(S_n-\mu) \le z\right] = \Phi\left(\frac{z}{\sigma}\right) ,$$

My Question: I am having a hard time understanding what is means when they say the above convergence happens with rate $\sqrt{n}$.

The notion of rate implies that it characterizes how fast something converges, but I am failing to see in the equation above where the slowness or the "fastness" is coming in. Can someone help me see this?

• One knows that $\frac1nS_n\to\mu$ and the rate of this convergence is described by the fact that $\frac1nS_n-\mu=\frac1{\sqrt{n}}Z_n$ where $Z_n\to Z$ in distribution with $Z$ centered normal. Thus the error term $\frac1nS_n-\mu$ is roughly $\frac1{\sqrt{n}}Z$, which is clearly of order $\frac1{\sqrt{n}}$. (Note that your question confuses $S_n$ with $\frac1nS_n$ at some places, it seems.)
– Did
May 7, 2017 at 9:00
• My feel is like: By WLLN, $S_n$ converge to $\mu$ in probability, i.e. the difference is converge to $0$. And $\sqrt{n}$ will goes to infinity as $n$ goes to infinity. When you multiply them together, it does not explode - it converge in distribution to a bounded random variable. So if you further multiply it by a higher order of $n$, it will explode. Is it what you want?
– BGM
May 7, 2017 at 9:04
• @BGM - You need to divide ${S_n}-n\mu$ by $\sqrt{n}$, or multiply $\frac{S_n}{n}$ or $\frac{S_n}{n}-\mu$ by $\sqrt{n}$ May 7, 2017 at 9:09

1 Answer

The mean of $S_n$ is $n\mu$ and the variance of $S_n$ is $n\sigma^2$ so the standard deviation of $S_n$ is $\sqrt{n}\sigma$

This has the consequence that $\dfrac{S_n}{n}$ has mean $\mu$ and variance $\dfrac{\sigma^2}{n}$ and standard deviation $\dfrac{\sigma}{\sqrt{n}}$. I suspect that this is what led to your phrase "convergence happens with rate $\sqrt{n}$"

So defining $T_n = \dfrac{S_n-n\mu}{\sqrt{n}\sigma}$, you can find $T_n$ has mean $0$ and variance $1$ and standard deviation $1$

What the central limit theorem says is that this converges in distribution to a standard normal as $n$ increases, i.e

$$\lim_{n\to\infty} \Pr\left[T_n \le t\right] = \Phi\left(t\right)$$ or $$\lim_{n\to\infty} \Pr\left[\dfrac{S_n-n\mu}{\sqrt{n}\sigma} \le t\right] = \Phi\left(t\right)$$ or $$\lim_{n\to\infty} \Pr\left[\sqrt{n}\dfrac{\frac{S_n}{n}-\mu}{\sigma} \le t\right] = \Phi\left(t\right)$$ and letting $z=t\sigma$ $$\lim_{n\to\infty} \Pr\left[\dfrac{S_n-n\mu}{\sqrt{n}} \le z\right] = \Phi\left(\frac{z}{\sigma}\right)$$ or $$\lim_{n\to\infty} \Pr\left[\sqrt{n}\left(\frac{S_n}{n}-\mu\right)\le z\right] = \Phi\left(\frac{z}{\sigma}\right)$$

which is not quite what you have written, as it is a description of the behaviour of $\frac{S_n}{n}$