# sum of squares of dependent gaussian random variables

Ok, so the Chi-Squared distribution with n degrees of freedom is the sum of the squares of n independent gaussian random variables.

The trouble is, my gaussian random variables are not independent. They do however all have zero mean and the same variance. Supposing I have a covariance matrix - which again is not a diagonal matrix because they aren't independent, but all the elements along the diagonal are equal to each other because they have the same variance, and in fact the covariance matrix is a symmetric toeplitz matrix (and I'm not saying that this is important to the solution if there is one, but if it's a necessary property to get anywhere, by all means use that fact) - is there some way to decompose this sum of squares of these gaussian random variables into perhaps a sum of chi-squared random variables and possibly gaussian random variables? In other words, I can't directly just square them all and add them together and call it a chi squared distribution because a chi squared distribution is a sum of independent gaussian squares, and they aren't independent.

I know how to find a linear transformation of the gaussian random variables which are n independent gaussians, but that's no help because they aren't the things being squared, you see.

-
Do you know (or can safely assume) that all your dependent normal variables, as a vector, is multinormal? In that case a solution should be possible! –  kjetil b halvorsen Jul 13 '13 at 16:52

## 1 Answer

Lets assume you have $X=(X_1, \dots, X_n)$ a random vector with multinormal distribution with expectation vector $\mu$ and covariance matrix $\Sigma$. We are interested in the quadratic form $Q(X)= X^T A X = \sum \sum a_{ij} X_i X_j$. Define $Y = \Sigma^{-1/2} X$ where we are assuming $\Sigma$ is invertible. Write also $Z=(Y-\Sigma^{-1/2} \mu)$, which will have expectation zero and variance matrix the identity.

Now $$Q(X) = X^T A X= (Z+\Sigma^{-1/2} \mu)^T \Sigma^{1/2}A\Sigma^{1/2} (Z+\Sigma^{-1/2} \mu).$$ Use the spectral theorem now and write $\Sigma^{1/2}A \Sigma^{1/2} = P^T \Lambda P$ where $P$ is an orthogonal matrix and $\lambda$ is diagonal with positive diagonal elements. Write $U = P^T Z$ so that $U$ is multivariate normal with identity covariance matrix and expectation zero.

We can compute $$Q(X) = (Z+\Sigma^{-1/2} \mu)^T \Sigma^{1/2}A\Sigma^{1/2} (Z+\Sigma^{-1/2} \mu)$$ $$= (U+b)^T P^T \Sigma^{1/2}A \Sigma^{1/2} P (U+b)$$ where now $b = P^T \Sigma^{-1/2} \mu$ so $$Q(X) = X^T A X = \sum_{j=1}^n \lambda_j (U_j+b_j)^2$$ In your case, $\mu=0$ so $b=0$ so your quadratic form is a linear combination of independent chi-square variables, each with one degree of freedom.

If you want to work numerically with that distribuion, there is an CRAN package (that is, package for R) implementing it, called CompQuadForm.

If you want (much ) more detail, there is a book dedicated to the topic, Mathai & Provost: "Quadratic forms in random variables".

-
Wow, you are awesome. I had pretty much given up on anyone answering this. –  zortharg Jul 15 '13 at 3:14
What it actually is that I'm trying to do, is find the variance of the sum of the square change in the value of a random walk after a time of t over a certain range of time. So for instance, the average square change after 3 time units if there are 7 samples is ((x(3)-x(0))^2+(x(4)-x(1))^2+(x(5)-x(2))^2+(x(6)-x(3))^2) / 4 - and if the random walk has a variance of 1 per time unit, then the expected value of this sum is 3 - but what is its variance? And I THOUGHT I answered it yesterday using induction, but the results aren't matching the results of my Monte Carlo simulations. Any ideas? –  zortharg Jul 15 '13 at 3:26
First a fast question? How did you do the empirical estimation? As you present it, it seems like you did the different sums on the same random walk, but then they will not be independent replications, so you have no reason to expect they will converge on the expected value. –  kjetil b halvorsen May 9 at 12:35