1
$\begingroup$

I have a vector $X$ of random variables with mean $\mu$ and covariance matrix $\Sigma$.

I want to create a vector $Y$ of random variables that are uncorrelated, such that the original variables in $X$ are linear combinations of the variables in $Y$.

I remember vaguely that this requires diagonalizing $\Sigma$, but I am not sure about the details.

For concreteness, assume that:

$$ \Sigma = {1\over 36} \begin{bmatrix} 5 & -1 & -1 & -1 & -1 & -1 \\ -1 & 5 & -1 & -1 & -1 & -1 \\ -1 & -1 & 5 & -1 & -1 & -1 \\ -1 & -1 & -1 & 5 & -1 & -1 \\ -1 & -1 & -1 & -1 & 5 & -1 \\ -1 & -1 & -1 & -1 & -1 & 5 \end{bmatrix} $$

By diagonalizing $\Sigma$, I get matrices $C$ and $D$ such that $D$ is diagonal and $\Sigma = C D C^{-1}$:

$$ C = \left[\begin{matrix}1 & -1 & -1 & -1 & -1 & -1\\1 & 1 & 0 & 0 & 0 & 0\\1 & 0 & 1 & 0 & 0 & 0\\1 & 0 & 0 & 1 & 0 & 0\\1 & 0 & 0 & 0 & 1 & 0\\1 & 0 & 0 & 0 & 0 & 1\end{matrix}\right] \\ D= \left[\begin{matrix}0 & 0 & 0 & 0 & 0 & 0\\0 & \frac{1}{6} & 0 & 0 & 0 & 0\\0 & 0 & \frac{1}{6} & 0 & 0 & 0\\0 & 0 & 0 & \frac{1}{6} & 0 & 0\\0 & 0 & 0 & 0 & \frac{1}{6} & 0\\0 & 0 & 0 & 0 & 0 & \frac{1}{6}\end{matrix}\right] $$ How do I proceed from here - how do I express the vector $X$ as a function of a vector $Y$ where all variables in $Y$ are uncorrelated?

Is there maybe a different way to get uncorrelated variables from correlated ones?

$\endgroup$
  • $\begingroup$ I assume you mean stochastically independent, not linearly independent? $\endgroup$ – Epiousios Sep 23 '17 at 21:03
  • $\begingroup$ @1524 yes, fixed. $\endgroup$ – Erel Segal-Halevi Sep 23 '17 at 21:12
  • $\begingroup$ Assuming that $X$ is jointly normal, I would normally de-correlate $X$ by pre-multiplying it by $A$ s.t. $A\Sigma A^T$ is diagonal. Then the fact that for jointly normal varibles, uncorrelatedness implies independence would yield the desired vector of independent variables. But in your case $\Sigma$ does not have full rank (this is called a singular normal distribution) and I'm not sure how this changes the approach. $\endgroup$ – Epiousios Sep 23 '17 at 21:41
  • $\begingroup$ You mention $A\Sigma A^T$, but the diagonalization gives you $A\Sigma A^{-1}$ - how are they related? $\endgroup$ – Erel Segal-Halevi Sep 23 '17 at 22:16
  • $\begingroup$ Since $\Sigma$ is symmetric, it can be orthogonally diagonalized. That is, you can find an orthogonal matrix $A$ such that $A\Sigma A^{-1}$ is diagonal. Because $A$ is orthogonal, one has $A^{-1}=A^T$ $\endgroup$ – Epiousios Sep 24 '17 at 5:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.