A bunch of random variables $X_i$ are independently and identically distributed with expected value $\mu$ and variance $\sigma^{2}$. Let $Y_M = \sum_{i=1}^{M}X_{i}$.

When $M$ is large, can we say the distribution of $Y_M$ is approximately $\mathcal{N}(M\mu,M\sigma^2)$? Or can we only say $\frac{1}{\sqrt{M}\sigma}(Y_M-M\mu) \to \mathcal{N}(0,1)$?

  • 1
    $\begingroup$ $Y\to\mathcal N(\mu,M\sigma^2)$ does not make much sense in general. How do you formalize such a "limiting" process? $\endgroup$ – Cave Johnson Dec 22 '16 at 13:12
  • $\begingroup$ I think when $M\sigma^2 \to \infty$, it does not make sense. But when $M\sigma^2$ is limited, is that make sense?@CaveJohnson $\endgroup$ – EsJack Dec 22 '16 at 13:18
  • $\begingroup$ It is always $\frac{Y-E(Y)}{\sigma(Y)}\to {\mathcal N}(0,1)$. Here you see that $E(Y) = M \mu$ and $\sigma (Y) = \sqrt{M}\sigma$. $\endgroup$ – Gribouillis Dec 22 '16 at 13:18
  • $\begingroup$ Thx! @Gribouillis $\endgroup$ – EsJack Dec 22 '16 at 13:23
  • 6
    $\begingroup$ And now that you reposted essentially a duplicate of this question, with the rather transparent goal of getting a confirmation of your belief that one can "say the distribution of $Y_M$ is approximately" normal, and that you got this confirmation (although the statement is mathematically wrong), where does that leave us? $\endgroup$ – Did Dec 22 '16 at 16:28

I think the following result is true (see Serfling's approximation theorem mathematical statistics, where the proof of the following result is said to be easy) :

If $X_1,X_2,\dots,X_n,\dots,$ are i.i.d with mean $\mu$ and finite variance $\sigma^2$ and $S_n = \sum_{i=1}^{n} X_i$ and $Z_n \sim \text{N}(n\mu,n\sigma^2)$ then $$\lim_{n \to \infty} \sup_{t \in \mathbb{R}}| P(S_n \leq t ) - P(Z_n \leq t)| = 0.$$

So there is a sense in which you can say $Y \sim N(M\mu,M\sigma^2),$ the above result shows that for sufficiently large $M$ the distribution function of $Y$ is close to the distribution function of $\text{N}(M\mu.M\sigma^2)$ everywhere.

  • $\begingroup$ Yes this is nice, uniform convergence is insensitive to rescalings and translations on the $x$ axis, even if they depend on $n$. $\endgroup$ – Gribouillis Dec 22 '16 at 14:09
  • $\begingroup$ That really helps! Thanks a lot. $\endgroup$ – EsJack Dec 22 '16 at 14:14
  • 1
    $\begingroup$ The point here is that although there is no sense in the statement "$Y_n \to N(n\mu,n\sigma^2)$", there is sense in the statement "the CDF of $Y_n$ is close to that of $N(n\mu,n\sigma^2)$ in some norm for large $n$", which is really what one would like to have. And the latter is indeed true, and it follows readily from the CLT. $\endgroup$ – Ian Dec 22 '16 at 14:17
  • $\begingroup$ Sorry to bother you again. I want to find the original statement of the proposition you mentioned in Serfling's book since I want to cite it. But I cannot find the exact same one. Is it the statement on page 20? Thanks again. @Arin Chaudhuri $\endgroup$ – EsJack Dec 22 '16 at 15:00
  • 1
    $\begingroup$ Sorry but no, $S_n$ is not asymptotically normal, only $\frac1{\sqrt{n}}(S_n-n\mu)$ is. $\endgroup$ – Did Dec 22 '16 at 16:24

Lots of confusion seem to be due to the bad notations.

Let $Y = \sum_{i=1}^{M}X_{i}$.

It is incorrect to write $Y = \sum_{i=1}^{M}X_{i}$ because the left hand side has nothing to do with $M$ but the right hand side does. You should use the notation $Y_M = \sum_{i=1}^{M}X_{i}$ instead.

When $M$ is large, can we say $Y \to \mathcal{N}(M\mu,M\sigma^2)$?

No. The limiting distribution should have nothing to do with $M$.

Or we have to say $\frac{1}{\sqrt{M}\sigma}(Y-\mu) \to \mathcal{N}(0,1)$?

No. The correct version should be $$ \dfrac{1}{\sqrt{M}\sigma}(Y_M-M\mu) \to \mathcal{N}(0,1) $$ in distribution as $M\to\infty$, where you should assume that $0<\sigma^2<\infty$.

  • $\begingroup$ You misunderstood the question, the OP is asking if $Y_M$ is a good approximation to $N(M\mu,M\sigma^2)$. Your edits have made the question nonsensical. $\endgroup$ – Arin Chaudhuri Dec 22 '16 at 14:05
  • 2
    $\begingroup$ I used every single word originally given by OP. You edited OP's question in your own way and criticized that my answer does not answer his question, which is unfair. $\endgroup$ – Jack Dec 22 '16 at 14:13
  • $\begingroup$ I apologize for my intemperate comments. I think my edits reflect the spirit of the original question better, I will let the OP decide. $\endgroup$ – Arin Chaudhuri Dec 22 '16 at 14:15
  • $\begingroup$ Well. It is my mistake. I will pay more attention to the notations later.Thanks anyway $\endgroup$ – EsJack Dec 22 '16 at 14:15
  • 1
    $\begingroup$ I see no reason why this answer should be downvoted (and this comes from somebody of the opinion that downvotes are an integral part of the mse package...). $\endgroup$ – Did Dec 22 '16 at 16:23

It is always $\frac{Y-E(Y)}{\sigma(Y)}\to {\mathcal N}(0,1)$. Here you see that $E(Y) = M \mu$ and $\sigma (Y) = \sqrt{M}\sigma$, so you can say that $$ \frac{Y-M\mu}{\sqrt{M}\sigma}\to {\mathcal N}(0,1) $$ The meaning of this is that $$ P\left(a \le \frac{Y-M\mu}{\sqrt{M}\sigma} \le b\right) \to F(b) - F(a) $$ where $F$ is the cumulative distribution function of ${\mathcal N}(0,1)$.

This inequality can also be written as

$$P\left(M\mu-a\sqrt{M}\sigma\le Y\le M\mu+b\sqrt{M}\sigma\right)\to F(b) - F(a) $$ In this sense, $Y$ can be treated as if it followed ${\mathcal N}(M\mu, M\sigma^2)$.

  • $\begingroup$ But I still want to know whether $Y$ itself can be treated as normal distribution. I seems I do not get the point... $\endgroup$ – EsJack Dec 22 '16 at 13:34
  • 3
    $\begingroup$ See my edit above. You can treat it as such as long as you don't add results of your own that are not given by the theorem. The theorem's conclusion is a precise statement. $\endgroup$ – Gribouillis Dec 22 '16 at 13:42
  • $\begingroup$ Thx for your explain! $\endgroup$ – EsJack Dec 22 '16 at 13:47

Not the answer you're looking for? Browse other questions tagged or ask your own question.