# Are two orthogonal linear transformations of the same random Gaussian vector independent?

Major revision:

I want to prove/disprove the following claim:

If $$e = (e_{1},\dots, e_{n})$$ is a vector of independent random variables, (each $$e_{i}$$ is normally distributed), and $$v_{1}, \dots v_{n} \in N^{n}$$ are n orthogonal vectors, then $$\langle v_{1}, e \rangle, \dots, \langle v_{n}, e \rangle$$ are independent.

And I'm a but puzzled. On one hand, it seems like this question supports a positive answer to this claim. The covariance matrix $$C$$ of $$e$$ is scalar, and the vectors $$v_{1},\dots, v_{n}$$ define a matrix A such that $$A\cdot A^{T}$$ is diagonal, so according to the explanation there $$ACA^{T}$$ is diagonal, and independency should follow.

On the other hand, this claim fails hard on toy example, when $$e$$'s values are drawn from discrete uniform distribution. For example, take $$e = (e_{1},e_{2})$$ to be vector of two uniform variables between 0 and p, and $$v_{1} = (1, 1), v_{2} = (1, -1)$$. Clearly if $$\langle v_{1}, e \rangle = 2p$$ then $$\langle v_{2}, e \rangle = 0$$.

What am I missing? I can't see the problem with the proof regarding normal distribution, nor find a reason that the same proof wouldn't work on the uniform distribution (where its obviously a false claim due to my counterexample)

Thanks!

• Normal distributions are fairly special and behave well where other simple distributions may not. As a simple example: any linear combination of independent normal variables is normally distributed, but this result fails catastrophically for uniform variables. So: trust your reading of the proof, and don't fret about the purported counterexample. – Aaron Montgomery Sep 24 '19 at 13:18
• But where the "the covariance matrix is diagonal, so the variables are independent" even use the fact that this is normal distribution? I don't see why this argument can't be applied as is to uniform distribution... – Bartolinio Sep 24 '19 at 13:31
• Although it wasn't explicit about it, the quoted statement does not apply in general and is particular to normal variables. If $X_1, X_2$ are independent and uniform on $[0,1]$, then $X_1 + X_2$ and $X_1 - X_2$ are uncorrelated (which is easy to show) but dependent (which you correctly argued). The fact that it does work for normal variables is a theorem that requires a proof. See, for instance, section 3 of cs229.stanford.edu/section/gaussians.pdf – Aaron Montgomery Sep 24 '19 at 13:50
• Thank you very much! – Bartolinio Sep 24 '19 at 14:11

In general: if $${\bf x}=(x_1,\cdots x_n)'$$ is uncorrelated (meaning that the elements $$x_i$$ are pairwise uncorrelated), and if $$A$$ is a $$n \times n$$ orthogonal matrix, then it's easy to prove that the variable
$${\bf y} = A {\bf x}$$
is also uncorrelated. That is, if $$C_{\bf x}$$ (covariance matrix of $${\bf x}$$) is diagonal, so is $$C_{\bf y}$$.
Now, if additionally $$x_i$$ are gaussian variables, then the components of $${\bf x}$$ are not only uncorrelated but independent. The result above implies that the variable $${\bf y} = A {\bf x}$$ is uncorrelated; but, because it's also joinly gaussian, then it's also independent.
Now if instead $$x_i$$ are just independent (hence uncorrelated) but not gaussian, then all we can say about $${\bf y} = A {\bf x}$$ is that $$y_i$$ are uncorrelated. They are not, in general, independent. As your example illustrates.