Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

(Sorry ahead of time, I don't know LaTeX...)

I've been studying some basic probability theory recently, and I'm having a little trouble understanding why the LLNs and CLT aren't contradictory. Given IID variables {X1,X2,...} each with mean M and variance V, both LLNs seem to say that the average of the first N variables in this sequence approaches M as N approaches infinity. The CLT seems to say that as N approaches infinity, the distribution of this average approaches a normal distribution with mean M and variance V.

The problem I'm having is that it seems like the distribution of the average should converge to something like a discrete variable with a PMF like 1 at M and 0 everywhere else. This is because the strong LLN says the the average MUST be M as N approaches infinity. Instead, the normal distribution given by the CLT seems to say that there's of chance of the average not being M as N approaches infinity, which seems to contradict the strong LLN.

I realize I have a flaw in my logic, since a lot of really smart people came up with these theorems :P but I'd just like to know where I'm going wrong. I'm assuming it has something to do with the differences between convergences of probability, distribution, etc, but I don't know too much about those so if someone could explain it clearly I'd really appreciate it.

Thanks!

share|improve this question

2 Answers 2

up vote 4 down vote accepted

The problem is that you left out the scaling. The average does converge to a constant: what converges to a normal distribution is $$\left(\text{average} - M\right)\sqrt{n}$$

share|improve this answer
    
If the (average-M) part is converging to 0, won't the $\sqrt{n}$ part not matter? –  Rotfuchs Jun 21 '13 at 19:37
    
Of course it will matter. $0 \times \infty$ is an indeterminate form. –  Robert Israel Jun 22 '13 at 21:50
    
I see, sorry about that. Thanks for the help! –  Rotfuchs Jun 23 '13 at 5:32

Heres my 5 cents.

Central limit theorem states that:

If we have $X_1,X_2,\dots$ iid with mean $\mu$ and variance $\sigma^2$ then $$U_n=\frac{1}{\sqrt{n\sigma ^2}} \sum _{i=1}^n (X_i - \mu) \to N(0,1)$$ in distribution.

Law of large numbers states that:

If $X_1,X_2,\dots$ are i.i.d with mean $\mu$ then $$S_n=\frac{1}{n}\sum_{i=1}^n X_i \to \mu $$ a.s.

Returning to the CLT we can see it also shows that (since $\frac{1}{\sqrt n}=\frac{\sqrt n}{\sqrt n \sqrt n}$): $$\sqrt{n}\cdot (\frac{1}{n} \sum _{i=1}^n (X_i - \mu)) \to N(0,\sigma ^2)$$

So you can see that CLT "works" a bit slower which is where the difference lies.

Convergence a.s. is much stronger than convergence in distribution, so you are right - had the scaling been the same it would imply convergence in distribution to a degenerate, that is almost surely constant random variable. In the case above $S_n \to X$ in distribution where $P(X=\mu)=1$

Maybe you can even now use CLT to say something about how fast the LLN convergence is?

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.