I'm taking a probability class and my prof used the following theorem IIRC.

Let $g\sim\mathcal{N}(\mu,\Sigma)$ where $\Sigma$ is diagonal( I don't know if this condition is necessary) and $\langle u,v\rangle=0$, then $\langle g,u\rangle$ and $\langle g,v\rangle$ are independent.

Is this correct? If so, how to prove this? I believe the following is a special case of the theorem: Are the random variables $X + Y$ and $X - Y$ independent if $X, Y$ are distributed normal? I tried to use the same technique to prove the theorem but got stuck.

  • $\begingroup$ If $\color{blue}{ u^T \Sigma v = 0}$, then you can show that $u^T g$ and $v^T g$ are independent. It won't generally hold if just $u^T v = 0$. Hint for showing this: write $z := (u^Tg, v^Tg)^T$ in the form $Ag$ for some matrix $A$, and use that the covariance matrix of $Ag$ is $A\Sigma A^T$. Also recall facts about linear transformations of normal random vectors and when such a vector has independent components based on its covariance matrix). $\endgroup$ – Minus One-Twelfth Feb 17 '19 at 2:54
  • $\begingroup$ So actually $\Sigma$ doesn't need to be diagonal? $\endgroup$ – Yihan Zhou Feb 17 '19 at 5:57
  • $\begingroup$ Correct! All that matters is that $u^T \Sigma v = 0$. $\endgroup$ – Minus One-Twelfth Feb 17 '19 at 5:59

Elaborating on Minus One-Twelfth's comment:

The pair $(g^\top u, g^\top v)$ is jointly normal. (Why?)

Thus independence is equivalent to $\text{Cov}(g^\top u, g^\top v) = 0$. The covariance is $$\begin{align}\text{Cov}(g^\top u, g^\top v) &= E[(g^\top u - E[g^\top u])(g^\top v - E[g^\top v])] \\ &= E[((g - \mu)^\top u)((g - \mu)^\top v)] \\ &= E[u^\top (g - \mu)(g-\mu)^\top v] \\ &= u^\top E[(g-\mu)(g-\mu)^\top] v \\ &= u^\top \Sigma v. \end{align}$$

If $\Sigma$ is a multiple of the identity matrix and if $\mu = 0$, then independence is equivalent to $u^\top v = 0$. However, in general you have to use the above expression $u^\top \Sigma v$.

  • $\begingroup$ Why is it jointly normal? Is this a property of the multivariate Gaussian variable? I know that $g^Tu$ and $g^Tv$ are both normal. $\endgroup$ – Yihan Zhou Feb 17 '19 at 3:36
  • $\begingroup$ @YihanZhou One way to check that it is jointly normal is to check that any linear combination $a(g^\top u) + b (g^\top v)$ is [univariate] normal. Indeed, any such linear combination is itself a linear combination of the components of $g$. $\endgroup$ – angryavian Feb 17 '19 at 3:40
  • $\begingroup$ Why $E[(u^Tg)(g^Tv)]=u^T\Sigma v$? Sorry I'm not stat major and don't know much about multivariate Gaussian. $\endgroup$ – Yihan Zhou Feb 17 '19 at 4:04
  • $\begingroup$ @YihanZhou See my edit; I made a mistake earlier. $\endgroup$ – angryavian Feb 17 '19 at 4:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.