# Sum of Product of Normal Random Variables

Consider a collection of $n$ i.i.d.normal random variables $X_i \sim \mathcal{N}(\mu, \sigma^2)$, $i=1,\ldots,n$. I'm trying to compute the distribution of $$\sum_{i=1}^n \sum_{j=i+1}^n X_i X_j.$$ It seems that each $X_i X_j$ has the normal product distribution, but the sum is elusive for me.

Update based on the answer from @Robert Israel:

Transform $\bf Z$ by an orthogonal matrix...

Let's say ${\bf Z} \in \mathbb{R}^2$ and let the orthogonal matrix be $$A = \begin{pmatrix} \frac{1}{\sqrt 2} & -\frac{1}{\sqrt 2} \\ \frac{1}{\sqrt 2} & \frac{1}{\sqrt 2} \end{pmatrix}.$$ Applying this transformation gives $$A {\bf Z} = \begin{pmatrix} \frac{1}{\sqrt 2}Z_1 -\frac{1}{\sqrt 2}Z_2 \\ \frac{1}{\sqrt 2}Z_1 +\frac{1}{\sqrt 2}Z_2 \\ \end{pmatrix}.$$ Since we have a ${\bf e}^T {\bf Z}$ term, apply this to the transformed ${\bf e}$: $$\begin{pmatrix} 1 & 1 \end{pmatrix} \begin{pmatrix} \frac{1}{\sqrt 2}Z_1 -\frac{1}{\sqrt 2}Z_2 \\ \frac{1}{\sqrt 2}Z_1 +\frac{1}{\sqrt 2}Z_2 \\ \end{pmatrix} = \sqrt 2 Z_1.$$ I believe @Robert Israel wrote this as $\sqrt n W = \sqrt n Z_1$, and mentioned

Thus $W$ is the component of ${\bf Z}$ in the direction of the unit vector $n^{−1/2}{\bf e}$, while the sum of the squares of the components for the other $n−1$ basis vectors in an orthonormal basis is $U$.

First, it seems that from my example, $W$ is just the first component of ${\bf Z}$, namely $Z_1$, so I'm not sure how this is also the first component of ${\bf Z}$ in the direction of $n^{−1/2}{\bf e}$. Secondly, I still don't quite understand the sum of squares of the other basis components becoming $U$.

• Why did you accept the answer below since you "don't quite understand (the) change of basis argument" in it? – Did Aug 4 '16 at 20:23
• @Did I verified his answer numerically, so I believe it to be correct. I'd like to understand it completely, and twice asked for elaboration, but he did his part in providing a correct answer. Would you care to elaborate? :) – bcf Aug 4 '16 at 20:56
• Me? I got @Robert's explanation, so... – Did Aug 4 '16 at 21:19

Your sum is $$S_n = \sum_{i=1}^n \sum_{j=i+1}^n X_i X_j = \dfrac{1}{2}{\bf X}^T C \bf X$$ where $C$ is the $n \times n$ matrix with all off-diagonal entries $1$ and diagonal entries $0$. We can write $C = e e^T - I$ where $e = [1,\ldots,1]^T$, and $\bf X = \mu \bf e + \sigma \bf Z$ where $\bf Z$ is multivariate normal with mean $0$ and variance $I$. Then

$$S_n = \dfrac{n(n-1)}{2} \mu^2 + (n-1) \mu \sigma {\bf e}^T {\bf Z} + \dfrac{\sigma^2}{2} \left(({\bf e}^T {\bf Z})^2 - {\bf Z}^T {\bf Z}\right)$$

Changing to an orthonormal basis where the first basis element is $n^{-1/2} \bf e$, this can be written as

$$S_n = \dfrac{n(n-1)}{2} \mu^2 + n^{1/2} (n-1) \mu \sigma W + \dfrac{\sigma^2(n-1)}{2} W^2 - \dfrac{\sigma^2}{2} U$$

where $W$ and $U$ are independent, $W$ is standard normal and $U$ is $\chi^2$ with $n-1$ degrees of freedom.

EDIT:

In the case $n=2$, you can take the orthogonal matrix $$M = \pmatrix{1/\sqrt{2} & -1/\sqrt{2}\cr 1/\sqrt{2} & 1/\sqrt{2}}$$ so that $${\bf e} = \sqrt{2} M \pmatrix{1\cr 0\cr}, \ {\bf Z} = M \bf W$$ where $\bf W$ is again multivariate normal with mean $0$ and variance $I$. Then $${\bf e}^T {\bf Z} = \sqrt{2}\; [1\ 0]\; M^T M {\bf W} = \sqrt{2} W_1$$ where $W_1$, the first entry of $\bf W$, is standard normal, and $$({\bf e}^T {\bf Z})^2 - {\bf Z}^T {\bf Z} = 2 W_1^2 - {\bf W}^T M^T M {\bf W} = 2 W_1^2 - {\bf W}^T {\bf W} = W_1^2 - W_2^2$$ $W_1$ is what I called $W$ above, and $W_2^2 = U$.

• Thanks, but would you comment more on changing to an orthonormal basis? Do you mean, change the basis of $C$? – bcf Jun 8 '16 at 13:03
• Change the basis of the space where $\bf Z$ lives. – Robert Israel Jun 8 '16 at 15:57
• Would you elaborate? It seems that $Z \in \mathbb{R}^n$; what does it mean to change the basis of a single vector? – bcf Jun 8 '16 at 21:24
• You transform $\bf Z$ by an orthogonal matrix (which preserves the property of being multivariate normal with mean $0$ and covariance matrix $I$). Thus $W$ is the component of $\bf Z$ in the direction of the unit vector $n^{-1/2} \bf e$, while the sum of the squares of the components for the other $n-1$ basis vectors in an orthonormal basis is $U$. – Robert Israel Jun 8 '16 at 22:21
• I'm actually still struggling to understand this argument. I updated my question to show what I am thinking, and exactly where I am still thrown off. Would you mind trying to explain a bit more in detail? – bcf Aug 5 '16 at 12:15