Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

This is a two-part question.

Suppose I am drawing random variables $X_i\sim A$, $1\leq i \leq n$ where $A$ is a zero-mean, finite variance $\sigma_A^2$, symmetric probability distribution having finite fourth moment $\mathbb{E}(X^4)$ over the support of real number line. Assume that for all $(i,j)$, $X_i$ and $X_j$ are independent. I am interested in approximating the distribution of the sum of squares $\sum_{i=1}^nX_i^2$ with the normal distribution, for very large $n$. In the second part of the question I relax the assumption that $X_i$'s are identically-distributed, but keep same conditions on each $A_i$.

By Central Limit Theorem (CLT), the sum of these i.i.d. random variables, the random variable $$\frac{\sum_{i=1}^nX_i}{\sqrt{n\sigma_A^2}}\xrightarrow{D}\mathcal{N}(0,1)$$

where $\xrightarrow{D}$ denotes convergence in distribution. Thus, I can approximate the distribution of the sum by $\mathcal{N}(0,n\sigma_A^2)$ for large enough $n$.

The first part of my question is: what distribution can I use to approximate the sum of the squares of a large number of these i.i.d. random variables $\sum_{i=1}^n X^2_i$? Do a function of it converge to a standard Gaussian in distribution (i.e. given large enough, possibly infinite $n$)? I understand that if $A$ is a Gaussian, then $\frac{1}{\sigma_A^2}\sum_{i=1}^n X^2_i\sim\chi^2(n)$, which can be approximated by $\mathcal{N}(n\sigma_A^2,2n\sigma_A^4)$ for very large $n$ using the asymptotic properties of chi-squared distribution (where $\sigma_A^4$ denotes squared variance.) But what happens when $A$ has the nice properties described above, but is not necessarily Gaussian?

My intuition tells me that that it should converge to a Gaussian, since we are still dealing with a sum of random variables with finite mean and variance. But I'm not sure how to prove that or characterize the distribution in terms of $n$ and $\sigma_A^2$.

Second part of the question is a further generalization on this topic. Now suppose $X_i\sim A_i$ are non-identically distributed. They are all still independent, and $A_i$ are still zero-mean and symmetric, but they all have different finite variances $\sigma_i^2$, and may have a different form. Since the means and variances are finite, Lindeberg's condition is met, which assures us that CLT holds for the $\frac{\sum_{i=1}^n X_i}{\sqrt{\sum_{i=1}^n\sigma_i^2}}\xrightarrow{D}\mathcal{N}(0,1)$. However, again, I am wondering what happens with the sum of squares $\sum_{i=1}^n X_i^2$. Is there a function of it that converges to a nice random variable such as Gaussian in distribution (i.e. for an appropriately large $n$, possibly infinite, does it look Gaussian) If so, to what distribution does it converge to and how can one characterize both the distribution and the function of $\sum_{i=1}^n X_i$ in terms of $n$ and $\sigma_i^2$, and possibly $\mathbb{E}(X_i^4)$? Is the result more attainable if each $A_i$ is a zero-mean Gaussian with variance $\sigma_i^2$?

Again, my intuition tells me that a function of $\sum_{i=1}^n X_i^2$ should converge to a Gaussian, since again we are dealing with the sum of random variables with finite means and variances, which should meet Lindeberg's condition... but is there a proof and how to characterization of this distribution in terms of $n$ and $\sigma_i^2$?

EDITS: I have changed the question after @Michael Hardy answered the first part for me. The second part is still open...

share|improve this question

2 Answers 2

up vote 5 down vote accepted

I have a certain degree of discomfort with the expression $$\sum_{i=1}^n X^2_i\xrightarrow{D}\mathcal{N}(n,2n\sigma_A^4)$$ since $n$ appears on both sides. If one takes a limit as $n\to\infty$, one gets something that does not depend on $n$.

When one says $$\sum_{i=1}^n X_i\sim\mathcal{N}(0,n\sigma_A^2),$$ it has to mean that $$ \frac{1}{\sigma_A\sqrt{n}}\sum_{i=1}^n X_i $$ converges in distribution to $\mathcal{N}(0,1)$ as $n\to\infty$, and no $n$ appears in the expression "$\mathcal{N}(0,1)$", which is the limit.

Since $\mathbb{E}(X_i) = 0$, we have $\sigma_A^2=\operatorname{var}(X_i)=\mathbb{E}(X_i^2)$, and $$ \operatorname{var}(X^2) = \mathbb{E}(X^4) - \sigma_A^4. $$

So if this last quantity happens to be finite then the central limit theorem tells us that $$ \frac{\sum_{i=1}^n (X_i^2 - \sigma_A^2) }{\sqrt{n}\sqrt{\mathbb{E}(X^4) - \sigma_A^4}} $$ converges in distribution to $\mathcal{N}(0,1)$ as $n\to\infty$.

share|improve this answer
    
The words "if this last quantity happens to be finite" are not vacuous. THere are some probability distributions for which $\mathbb{E}(X^2)<\infty$ and $\mathbb{E}(X^4)=\infty$. For example, Student's t-distribution with a sufficiently small number of degrees of freedom, or the Pareto distribution with a small enough index. –  Michael Hardy Nov 5 '11 at 16:30
    
Thank you for your answer. I understand your discomfort with having $n$ on both sides of the expression. Basically, as @Sasha guessed, I am trying to approximate a sum of very large number of $X_i^2$, and am trying to figure out whether I can use a Gaussian for it, and under which conditions. Looks like I can use it as long as the fourth moment of $X_i$ is finite (I think I can argue that in the problem I am trying to solve.) I will re-write my question to clarify that. –  M.B.M. Nov 6 '11 at 2:08
    
Also, what happens when $X_i$'s are non-identical but independent? I will also add the assumption that the fourth moment of each $A_i$ is finite (and known.) This is the second part of my question, and probably the more important part. –  M.B.M. Nov 6 '11 at 2:19
    
There are numerous variants of the central limit theorem, and some allow the distributions not to be identical, and some of them relax the assumption of independence, and some of them allow unequal variances. I think if the sum of the variances is finite one might be in trouble. There's a book on probability by Grimmett & Stirzaker that I seem to recall has stuff on this; I'll take a look. I believe there are other books that concentrate much more on this topic........ –  Michael Hardy Nov 6 '11 at 3:56
    
Btw, what do you mean by "if the sum of the variances is finite one might be in trouble"? Finite over finite $n$? Because, if variance for each $A_i$ is finite, than over $n\rightarrow\infty$, the sum of variances would also be infinite (and contribution from each variance would be small, which I think is how Lindeberg condition works). Anyway, if you could look that up, I would appreciate the help. Now, would case where $A_i$ are zero-mean Gaussians with different variances $\sigma_i^2$ be easier to characterize? Even that result would be very useful... –  M.B.M. Nov 6 '11 at 4:45

The central limit theorem states that, for i.i.d. sequence $X_i$ random variables with mean $\mathbb{E}(X)$ and variance $\operatorname{Var}(X)$, the random variable sequence $Z_n = \frac{1}{\sqrt{n \operatorname{Var}(X)}} \left( \sum_{i=1}^n X_i - n \mathbb{E}(X) \right)$ converges in distribution to the standard normal distribution.

CLT makes no statement about distribution of $Z_n$ for finite $n$, though.

share|improve this answer
    
Sorry, I guess I am kind of sloppy in my notation. By "large enough" $n$ I mean "as large as you want." What I am after is convergence in distribution for $\sum_{i=1}^n X_i$. I'll edit the question to be more precise. Unfortunately, your answer just restates the CLT... –  M.B.M. Nov 5 '11 at 5:44
1  
@Bullmoose By weak law of large numbers $Y_n = \sum_{i=1}^n X_i^2$ will diverge almost surely as $n \to \infty$. For finite $n$ the cumulative distribution of $Y_n$ is well approximated by that of $\mathcal{N}( n \mathbb{E}(X^2), n (\mathbb{E}(X^4) - \mathbb{E}(X^2)^2) )$. –  Sasha Nov 5 '11 at 5:55
    
could you please expand on the last statement about $Y_n$? How can one prove that, or find the proof? Also, would $\frac{\sum_{i=1}^nX_i^2}{n}$ diverge? –  M.B.M. Nov 5 '11 at 6:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.