I'm learning about the LLN and CLT for the first time and I'm having some trouble. I've read through other posts but I have a quirky (likely dumb) question about denominators...

The Strong LLN says for $X_1,...,X_n$ iid with common mean $\mu$,

$$P \left( \lim_{n \rightarrow \infty} \frac{X_1 + ... + X_n }{n} = \mu \right) = 1$$

My question 1: Instead of dividing by $n$, is it true that if I divide by a function of $n$, say $f(n) = Cn^k$ where $k \ge 1$ then the limit will converge to some constant always? (not necessarily $\mu$)

$$P \left( \lim_{n \rightarrow \infty} \frac{X_1 + ... + X_n }{f(n)} = \text{some constant} \right) = 1$$

I believe this is true because $Var\left(\frac{S_n}{f(n)}\right) = \frac{n \sigma^2}{(f(n))^2}$ and as long as $f(n)$ grows as fast as $n$ then the variance will go to zero as $n$ goes to infinity.

My question 2: If instead of dividing by $n$ in the Strong LLN, if I divide by $ \sigma\sqrt{n}$ then is it true that

$$P \left( \lim_{n \rightarrow \infty} \frac{X_1 + ... + X_n }{\sigma \sqrt{n}} = \text{some constant} \right) \neq 1$$

and it won't "converge" to a constant because the RV $\frac{S_n}{\sigma \sqrt{n}}$ is a Normal RV with variance $1$.

My question 3: It seems to me what you divide $S_n$ by is very important and can lead to very different results like the LLN or CLT... is that a somewhat correct way of thinking?

Thank you!!

  • $\begingroup$ Please also quote the conditions on the random variables. $\endgroup$ – GNUSupporter 8964民主女神 地下教會 Oct 1 '18 at 20:49
  • $\begingroup$ Ah yes the $X_i$'s are all iid. I will edit it. $\endgroup$ – HJ_beginner Oct 1 '18 at 20:50
  • $\begingroup$ That's still incomplete. Take $\Omega = \Bbb{N}$ and $X_i = id_{\Bbb N}$ to see this. $\endgroup$ – GNUSupporter 8964民主女神 地下教會 Oct 1 '18 at 20:51
  • $\begingroup$ @GNUSupporter8964民主女神地下教會 hmmm... well I mean whatever conditions that they introduce for beginners when learning the LLNs... sorry, unfortuantely I don't know the notation in what you wrote, $X_i = id_{\Bbb N}$ - it seems $X_2 = 2d_{\Bbb N} \neq 3d_{\Bbb N} = X_3$ and so they are not iid but I think I just don't understand the notation $\endgroup$ – HJ_beginner Oct 1 '18 at 20:55
  • $\begingroup$ I mean the identity function on N. $X_i(w) = w$ for all w,i in N. $\endgroup$ – GNUSupporter 8964民主女神 地下教會 Oct 1 '18 at 20:57

Regarding 1 and 2: If you divide by $Cn^k$ then the limit will be $\mu/C$ (if $k=1$) or $0$ (if $k>1$). When $k<1$ the result will be $\infty$ with the same sign as $\mu$ unless $\mu=0$, in which case the situation is more complicated.

Regarding 3, yes: essentially you have different results when you measure deviation of the sample mean from the population mean in different scales relative to $n$. Under "typical" circumstances (when $X_i$ has a MGF existing in a neighborhood of $0$):

  • $O(1)$ scale fluctuations asymptotically vanish (this is LLN)
  • $O(n^{-1/2})$ scale fluctuations persist with probability strictly between $0$ and $1$ (this is CLT)
  • $O(1)$ scale fluctuations decay exponentially fast in probability (this is Cramer's theorem, a type of large deviation principle).
  • $\begingroup$ Thanks for your help. Before I learned about the LLN/CLT, I divided RVs like $S_n$ by constants all the time and it wasn't a big deal. The result would still be a RV and what the constant actually was would not dramatically change the outcome. But for LLN/CLT it seems what you divide $S_n$ by as $n$ goes to infinity has really big impact... $\endgroup$ – HJ_beginner Oct 1 '18 at 21:15
  • $\begingroup$ by the way should the third bullet point maybe be $O(n^k)$ for $k>1$? Again thanks for your help. $\endgroup$ – HJ_beginner Oct 1 '18 at 21:31
  • 1
    $\begingroup$ @HJ_beginner These are fluctuations of the sample mean around the population mean. The third bullet point is just a more precise version of the first one (but with stronger hypotheses being required): it gives a quantitative estimate of the convergence rate of $P(|\overline{X}-\mu|>\epsilon)$. $\endgroup$ – Ian Oct 1 '18 at 21:46
  • $\begingroup$ ahh thanks for the clarification!! $\endgroup$ – HJ_beginner Oct 1 '18 at 23:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.