4
$\begingroup$

Related to Confusion of central limit theory, and the fact that I just finished my first course in (Master's-level) graduate-level probability, which relates to this material.

The Central Limit Theorem states that if you have an iid sample $X_1, \dots, X_n$ with mean $\mu$ and variance $\sigma^2<\infty$, denoting $\bar{X}_n = \dfrac{1}{n}\sum_{i=1}^{n}X_i$, we have

$$\sqrt{n}(\bar{X}_n - \mu) \overset{d}{\to}\mathcal{N}(0, \sigma^2)$$ as $n \to \infty$. By Slutsky's theorem, since $\sigma = \sqrt{\sigma^2}$ is constant (let's assume in addition $\sigma \neq 0$), obviously $\sigma \overset{p}{\to}\sigma$, hence $$\dfrac{\bar{X}_n - \mu}{\sigma/\sqrt{n}}\overset{d}{\to}\dfrac{1}{\sigma}\cdot\mathcal{N}(0, \sigma^2) = \mathcal{N}(0, 1)$$ as $n \to \infty$. This justifies (to me) what is commonly what is done in intro stats classes: basically, if $n \geq 30$, if you want to, say, find $$\mathbb{P}(a \leq \bar{X}_n \leq b)$$ where $a$ and $b$ are usually finite, the idea is that when you standardize it as follows: $$\mathbb{P}(a \leq \bar{X}_n \leq b)\approx\mathbb{P}\left(\dfrac{a-\mu}{\sigma/\sqrt{n}} \leq \dfrac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} \leq \dfrac{b-\mu}{\sigma/\sqrt{n}}\right)$$ you can approximate $$\dfrac{\bar{X}_n - \mu}{\sigma/\sqrt{n}}$$ to be a $\mathcal{N}(0, 1)$ random variable if $n$ is "large enough;" the standard usually being $30$.

Now the question I have: this implies, then, that we can't necessarily assume $$\bar{X}_n\overset{d}{\to}\mathcal{N}\left(\mu, \dfrac{\sigma^2}{n}\right)$$ as $n \to \infty$. Is it correct that this isn't true?

For one thing, this makes no sense, as $n$ is still showing up in the normal distribution (in the variance) as $n \to \infty$. Obviously, Slutsky's theorem will not work (as far as I can tell).

But this seems to be contradicted by a select few websites:

https://onlinecourses.science.psu.edu/stat800/node/36

enter image description here

http://onlinestatbook.com/2/sampling_distributions/samp_dist_mean.html enter image description here

What am I missing?

Edit: The reason why I believe Slutsky's Theorem will not work is as follows: denote $Y_n = \dfrac{\bar{X}_n - \mu}{\sigma/\sqrt{n}}$ and let $F_{Z}$ denote the CDF of $Z \sim \mathcal{N}(0, 1)$. Then for all $y \in \mathbb{R}$ for which $F_{Y_n}$ is continuous (notice, particularly, that $y$ is a constant), $$\lim_{n \to \infty}F_{Y_n}(y) = F_{Z}(y)$$ but $$F_{\bar{X}_n}(x) = \mathbb{P}(\bar{X}_n \leq x) = \mathbb{P}\left(\dfrac{\sigma Y_n}{\sqrt{n}}+\mu \leq x\right) = \mathbb{P}\left(Y_n\leq\dfrac{x-\mu}{\sigma/\sqrt{n}}\right) = F_{Y_n}\left(\dfrac{x-\mu}{\sigma/\sqrt{n}}\right)\text{.}$$ Unfortunately, the argument $\dfrac{x-\mu}{\sigma/\sqrt{n}}$ is dependent on $n$, so Slutsky's Theorem won't help.

$\endgroup$
  • 4
    $\begingroup$ The first quote captures the essence of it: when $n$ is reasonably large ($30$ or more), you can approximate the sampling distribution of $\bar{x}$ with a normal as described. The second quote tries to say the same thing but goofs up and makes it mathematically wrong because as you point out: the sample size cannot appear in the limiting distribution. $\endgroup$ – yurnero Dec 22 '16 at 13:52
  • $\begingroup$ @yurnero Huh, that's interesting. I didn't realize that the two quotes were saying different things, but I see what you mean now. $\endgroup$ – Clarinetist Dec 22 '16 at 14:01
5
$\begingroup$

$$\bar{X}_n\overset{d}{\to}\mathcal{N}\left(\mu, \dfrac{\sigma^2}{n}\right)\tag{1}$$ as $n \to \infty$.

You don't need any deep mathematics to see why (1) is not true. Because the notations in (1) does not make sense at all.

Think about the following example. Note that you have the following limit $$ \lim_{n\to\infty}\frac{1}{1+n}=0. $$

On the other hand, for $n$ large enough, one can approximate $\dfrac{1}{1+n}$ by $\dfrac{1}{n}$ because $\lim_{n\to\infty}\biggr(\frac{1}{1+n}-\frac{1}{n}\biggr)=0$. However, one cannot say $$ \frac{1}{n+1}\to\frac{1}{n} \quad \text{as } n\to\infty.\tag{2}$$

$\endgroup$
  • 1
    $\begingroup$ Yes and no. One does have $\frac{1}{1+n}=O(\frac{1}{n})$ as $n\to\infty$. But it is not quite the point. The point is that the writing in (2) is the same as $$ \lim_{n\to\infty}\frac{1}{n+1}=\frac{1}{n}$$ which is completely wrong. $\endgroup$ – Jack Dec 22 '16 at 14:11
1
$\begingroup$

Nice question! If the problem is that the variance vanishes as $n \to \infty$, here's an intuitive explanation. Let's see it from this view point: Why does variance exists at all? Because we are uncertain about the random variable $X$. The variance of $\overline{X_n}$ is a measure of uncertainty that we have about $\overline{X_n}$, which obviously is generated from our uncertainty about $X$. From a statistician's viewpoint, the more data you can obtain, the less is the uncertainty. Here as $n$ increases, we obtain more data about $X$, and hence, as expected, the variance of $\overline{X_n}$ decreases. Now if $n$ approaches $\infty$ (but not equal to $\infty$, of course), the variance can be made arbitrarily small (but still positive!). In other words, the statistic $\overline{X_n}$ converges to $\mu$ with arbitrarily small uncertainty, i.e. for a pre-fixed $\delta > 0$, the quantity $P(|\overline{X_n}-\mu|>\delta)$ can be made arbitrarily small by choosing $n$ large enough. So the uncertainty, while cannot be fully conquered, can be controlled to arbitrary precision by obtaining more and more data.

NOTE THAT it might seem that the variance goes to zero and the distribution becomes degenerate. But that's NOT happening here! The graph of the probability density function may show a straight vertical line (unit mass) on the point $\mu$, but that's only in bare eyes. Instead, if you can look at it by changing the scale accordingly (like using a magnifying glass), you'll see that it's a normal bell-curve, which only look like a straight vertical line in our bare eyes.

$\endgroup$
1
$\begingroup$

Your notation is sloppy and confused me quite a while. If we put this way: $$ F_{\sqrt{n} (\overline{X}_n -\mu)} \overset{d}{\to} \mathcal{N}(0,\sigma^2) $$ implies $$ F_{\sqrt{n} (\overline{X}_n -\mu)} -\mathcal{N}(0,\sigma^2) \overset{d}{\to} 0 $$ implies (explained just below) $$ F_{\overline{X}_n -\mu} -\mathcal{N}(0,\sigma^2/n) \overset{d}{\to} 0 \quad ...[\spadesuit] $$ And then $$ F_{\overline{X}_n} -\mathcal{N}(\mu,\sigma^2/n) \overset{d}{\to} 0 $$ I.e., averaged $\overline{X}_n$ resembles, pointwise, the Gaussian with $\mu$ and $\sigma^2/n$. May be this just better captures what laymen mean by "oh, you see, this becomes close to Gaussian!"

The $[\spadesuit]$ follows by no means of difficult probability theory. The claim is pointwise: when 1st term approximates 2nd term for some $x$, then same is true when the prob. mass is scaled correspondingly, so we are comparing the very same density times $\sqrt{n}$ only now at $x/\sqrt{n}$ (more exactly $x$'s for which r.h.s. is conti., but in this short section everything at r.h.s. is conti.). The following (last) step is even easier, since we are just comparing the same thing with a shift $\mu$ w.r.t. $x$. I leave to the reader how it is said with lovely $\epsilon$ and $\delta$.

$\endgroup$
  • $\begingroup$ Yeah, I apologize for using $\mathcal{N}(0, \sigma^2)$ that way. I'll have to unlearn that habit from my course, but it unfortunately seems to be rather standard in the grad-level probability texts that I've seen. $\endgroup$ – Clarinetist Dec 22 '16 at 14:58
  • $\begingroup$ It's okay with me, but if you wrote that way in the final exam of "probability theory", I wonder how you pass it.... ;) At least when I took probability theory, the TA required fairly rigorous proof... $\endgroup$ – Aminopterin Dec 22 '16 at 15:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.