# Are the projections along orthogonal direction of multivariate normal distribution with diagonal covariance matrix independent?

I'm taking a probability class and my prof used the following theorem IIRC.

Let $$g\sim\mathcal{N}(\mu,\Sigma)$$ where $$\Sigma$$ is diagonal( I don't know if this condition is necessary) and $$\langle u,v\rangle=0$$, then $$\langle g,u\rangle$$ and $$\langle g,v\rangle$$ are independent.

Is this correct? If so, how to prove this? I believe the following is a special case of the theorem: Are the random variables $X + Y$ and $X - Y$ independent if $X, Y$ are distributed normal? I tried to use the same technique to prove the theorem but got stuck.

• If $\color{blue}{ u^T \Sigma v = 0}$, then you can show that $u^T g$ and $v^T g$ are independent. It won't generally hold if just $u^T v = 0$. Hint for showing this: write $z := (u^Tg, v^Tg)^T$ in the form $Ag$ for some matrix $A$, and use that the covariance matrix of $Ag$ is $A\Sigma A^T$. Also recall facts about linear transformations of normal random vectors and when such a vector has independent components based on its covariance matrix). – Minus One-Twelfth Feb 17 '19 at 2:54
• So actually $\Sigma$ doesn't need to be diagonal? – Yihan Zhou Feb 17 '19 at 5:57
• Correct! All that matters is that $u^T \Sigma v = 0$. – Minus One-Twelfth Feb 17 '19 at 5:59

Elaborating on Minus One-Twelfth's comment:

The pair $$(g^\top u, g^\top v)$$ is jointly normal. (Why?)

Thus independence is equivalent to $$\text{Cov}(g^\top u, g^\top v) = 0$$. The covariance is \begin{align}\text{Cov}(g^\top u, g^\top v) &= E[(g^\top u - E[g^\top u])(g^\top v - E[g^\top v])] \\ &= E[((g - \mu)^\top u)((g - \mu)^\top v)] \\ &= E[u^\top (g - \mu)(g-\mu)^\top v] \\ &= u^\top E[(g-\mu)(g-\mu)^\top] v \\ &= u^\top \Sigma v. \end{align}

If $$\Sigma$$ is a multiple of the identity matrix and if $$\mu = 0$$, then independence is equivalent to $$u^\top v = 0$$. However, in general you have to use the above expression $$u^\top \Sigma v$$.

• Why is it jointly normal? Is this a property of the multivariate Gaussian variable? I know that $g^Tu$ and $g^Tv$ are both normal. – Yihan Zhou Feb 17 '19 at 3:36
• @YihanZhou One way to check that it is jointly normal is to check that any linear combination $a(g^\top u) + b (g^\top v)$ is [univariate] normal. Indeed, any such linear combination is itself a linear combination of the components of $g$. – angryavian Feb 17 '19 at 3:40
• Why $E[(u^Tg)(g^Tv)]=u^T\Sigma v$? Sorry I'm not stat major and don't know much about multivariate Gaussian. – Yihan Zhou Feb 17 '19 at 4:04
• @YihanZhou See my edit; I made a mistake earlier. – angryavian Feb 17 '19 at 4:13