# Creating uncorrelated random variables using diagonalization

I have a vector $X$ of random variables with mean $\mu$ and covariance matrix $\Sigma$.

I want to create a vector $Y$ of random variables that are uncorrelated, such that the original variables in $X$ are linear combinations of the variables in $Y$.

I remember vaguely that this requires diagonalizing $\Sigma$, but I am not sure about the details.

For concreteness, assume that:

$$\Sigma = {1\over 36} \begin{bmatrix} 5 & -1 & -1 & -1 & -1 & -1 \\ -1 & 5 & -1 & -1 & -1 & -1 \\ -1 & -1 & 5 & -1 & -1 & -1 \\ -1 & -1 & -1 & 5 & -1 & -1 \\ -1 & -1 & -1 & -1 & 5 & -1 \\ -1 & -1 & -1 & -1 & -1 & 5 \end{bmatrix}$$

By diagonalizing $\Sigma$, I get matrices $C$ and $D$ such that $D$ is diagonal and $\Sigma = C D C^{-1}$:

$$C = \left[\begin{matrix}1 & -1 & -1 & -1 & -1 & -1\\1 & 1 & 0 & 0 & 0 & 0\\1 & 0 & 1 & 0 & 0 & 0\\1 & 0 & 0 & 1 & 0 & 0\\1 & 0 & 0 & 0 & 1 & 0\\1 & 0 & 0 & 0 & 0 & 1\end{matrix}\right] \\ D= \left[\begin{matrix}0 & 0 & 0 & 0 & 0 & 0\\0 & \frac{1}{6} & 0 & 0 & 0 & 0\\0 & 0 & \frac{1}{6} & 0 & 0 & 0\\0 & 0 & 0 & \frac{1}{6} & 0 & 0\\0 & 0 & 0 & 0 & \frac{1}{6} & 0\\0 & 0 & 0 & 0 & 0 & \frac{1}{6}\end{matrix}\right]$$ How do I proceed from here - how do I express the vector $X$ as a function of a vector $Y$ where all variables in $Y$ are uncorrelated?

Is there maybe a different way to get uncorrelated variables from correlated ones?

• I assume you mean stochastically independent, not linearly independent? – Epiousios Sep 23 '17 at 21:03
• @1524 yes, fixed. – Erel Segal-Halevi Sep 23 '17 at 21:12
• Assuming that $X$ is jointly normal, I would normally de-correlate $X$ by pre-multiplying it by $A$ s.t. $A\Sigma A^T$ is diagonal. Then the fact that for jointly normal varibles, uncorrelatedness implies independence would yield the desired vector of independent variables. But in your case $\Sigma$ does not have full rank (this is called a singular normal distribution) and I'm not sure how this changes the approach. – Epiousios Sep 23 '17 at 21:41
• You mention $A\Sigma A^T$, but the diagonalization gives you $A\Sigma A^{-1}$ - how are they related? – Erel Segal-Halevi Sep 23 '17 at 22:16
• Since $\Sigma$ is symmetric, it can be orthogonally diagonalized. That is, you can find an orthogonal matrix $A$ such that $A\Sigma A^{-1}$ is diagonal. Because $A$ is orthogonal, one has $A^{-1}=A^T$ – Epiousios Sep 24 '17 at 5:56