2
$\begingroup$

Major revision:

I want to prove/disprove the following claim:

If $e = (e_{1},\dots, e_{n})$ is a vector of independent random variables, (each $e_{i}$ is normally distributed), and $v_{1}, \dots v_{n} \in N^{n}$ are n orthogonal vectors, then $\langle v_{1}, e \rangle, \dots, \langle v_{n}, e \rangle$ are independent.

And I'm a but puzzled. On one hand, it seems like this question supports a positive answer to this claim. The covariance matrix $C$ of $e$ is scalar, and the vectors $v_{1},\dots, v_{n}$ define a matrix A such that $A\cdot A^{T}$ is diagonal, so according to the explanation there $ACA^{T}$ is diagonal, and independency should follow.

On the other hand, this claim fails hard on toy example, when $e$'s values are drawn from discrete uniform distribution. For example, take $e = (e_{1},e_{2})$ to be vector of two uniform variables between 0 and p, and $v_{1} = (1, 1), v_{2} = (1, -1)$. Clearly if $\langle v_{1}, e \rangle = 2p$ then $\langle v_{2}, e \rangle = 0$.

What am I missing? I can't see the problem with the proof regarding normal distribution, nor find a reason that the same proof wouldn't work on the uniform distribution (where its obviously a false claim due to my counterexample)

Thanks!

$\endgroup$
  • $\begingroup$ Normal distributions are fairly special and behave well where other simple distributions may not. As a simple example: any linear combination of independent normal variables is normally distributed, but this result fails catastrophically for uniform variables. So: trust your reading of the proof, and don't fret about the purported counterexample. $\endgroup$ – Aaron Montgomery Sep 24 '19 at 13:18
  • $\begingroup$ But where the "the covariance matrix is diagonal, so the variables are independent" even use the fact that this is normal distribution? I don't see why this argument can't be applied as is to uniform distribution... $\endgroup$ – Bartolinio Sep 24 '19 at 13:31
  • 1
    $\begingroup$ Although it wasn't explicit about it, the quoted statement does not apply in general and is particular to normal variables. If $X_1, X_2$ are independent and uniform on $[0,1]$, then $X_1 + X_2$ and $X_1 - X_2$ are uncorrelated (which is easy to show) but dependent (which you correctly argued). The fact that it does work for normal variables is a theorem that requires a proof. See, for instance, section 3 of cs229.stanford.edu/section/gaussians.pdf $\endgroup$ – Aaron Montgomery Sep 24 '19 at 13:50
  • 1
    $\begingroup$ Thank you very much! $\endgroup$ – Bartolinio Sep 24 '19 at 14:11
1
$\begingroup$

In general: if ${\bf x}=(x_1,\cdots x_n)'$ is uncorrelated (meaning that the elements $x_i$ are pairwise uncorrelated), and if $A$ is a $n \times n$ orthogonal matrix, then it's easy to prove that the variable

$${\bf y} = A {\bf x}$$

is also uncorrelated. That is, if $C_{\bf x}$ (covariance matrix of ${\bf x}$) is diagonal, so is $C_{\bf y}$.

Now, if additionally $x_i$ are gaussian variables, then the components of ${\bf x}$ are not only uncorrelated but independent. The result above implies that the variable ${\bf y} = A {\bf x}$ is uncorrelated; but, because it's also joinly gaussian, then it's also independent.

Hence your claim is correct.

Now if instead $x_i$ are just independent (hence uncorrelated) but not gaussian, then all we can say about ${\bf y} = A {\bf x}$ is that $y_i$ are uncorrelated. They are not, in general, independent. As your example illustrates.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.