Given $n$ i.i.d. random variables $\{X_1, X_2, \dots , X_n\}$, each with mean $M$ and variance $V$, both strong and week LLNs seem to say that the average of the $n$ random variables, $S_n = \frac{X_1 + X_2 + \dots + X_n }{n}$, approaches $M$, as $n \to \infty$. The CLT seems to say that, as $n \to \infty$, the distribution of this average $S_n$ approaches a normal distribution with mean $M$ and variance $V$.

The problem I'm having is that it seems like the distribution of the average should converge to something like a discrete variable with a PMF like $1$ at $M$ and $0$ everywhere else. This is because the strong LLN says the average must be $M$, as $n$ approaches infinity. Instead, the normal distribution given by the CLT seems to say that there's a chance of the average not being $M$, as $n$ approaches infinity, which seems to contradict the strong LLN.

Where's the flaw in my reasoning?


The problem is that you left out the scaling. The average does converge to a constant: what converges to a normal distribution is $$\left(\text{average} - M\right)\sqrt{n}$$


Heres my 5 cents.

Central limit theorem states that:

If we have $X_1,X_2,\dots$ iid with mean $\mu$ and variance $\sigma^2$ then $$U_n=\frac{1}{\sqrt{n\sigma ^2}} \sum _{i=1}^n (X_i - \mu) \to N(0,1)$$ in distribution.

Law of large numbers states that:

If $X_1,X_2,\dots$ are i.i.d with mean $\mu$ then $$S_n=\frac{1}{n}\sum_{i=1}^n X_i \to \mu $$ a.s.

Returning to the CLT we can see it also shows that (since $\frac{1}{\sqrt n}=\frac{\sqrt n}{\sqrt n \sqrt n}$): $$\sqrt{n}\cdot (\frac{1}{n} \sum _{i=1}^n (X_i - \mu)) \to N(0,\sigma ^2)$$

So you can see that CLT "works" a bit slower which is where the difference lies.

Convergence a.s. is much stronger than convergence in distribution, so you are right - had the scaling been the same it would imply convergence in distribution to a degenerate, that is almost surely constant random variable. In the case above $S_n \to X$ in distribution where $P(X=\mu)=1$

Maybe you can even now use CLT to say something about how fast the LLN convergence is?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.