# Uncorrelating random variables.

I was reading this answer, and the first sentence seemed more intuitive at first than after thinking through it:

If $\pmatrix{X\\ Y}$ is bivariate normal with mean $\pmatrix{0\\0}$ and covariance matrix $\Sigma=\pmatrix{1&\rho\\\rho&1}$, then $\pmatrix{U\\V}=\Sigma^{-1/2} \pmatrix{X\\Y}$ is bivariate normal with mean $\pmatrix{0\\0}$ and covariance matrix $\pmatrix{1&0\\ 0&1}.$ That is, $U$ and $V$ are independent, standard normal random variables.

At first I thought that $\Sigma^{-1/2}= \begin{pmatrix}1&&\frac{1}{\sqrt{\rho}}\\\frac{1}{\sqrt{\rho}}&&1\end{pmatrix}$.

## But this is clearly not the case as the answers so far explain.

This mistake corrected, I still would like to understand why:

$\frac{1}{2 \sqrt{1-\rho^2}}\begin{pmatrix}\sqrt{1-\rho}+\sqrt{1+\rho} & \sqrt{1-\rho}-\sqrt{1+\rho} \\ \sqrt{1-\rho}-\sqrt{1+\rho} & \sqrt{1-\rho}+\sqrt{1+\rho}\end{pmatrix}\begin{pmatrix}\mathbf X\\ \mathbf Y\end{pmatrix}$ manages to de-correlate $\bf X$ and $\bf Y$.

Or, $\frac{1}{2 \sqrt{1-\rho^2}}\begin{pmatrix}\sqrt{1-\rho}+\sqrt{1+\rho} & \sqrt{1-\rho}-\sqrt{1+\rho} \\ \sqrt{1-\rho}-\sqrt{1+\rho} & \sqrt{1-\rho}+\sqrt{1+\rho}\end{pmatrix} \mathbf A^T$ with $\mathbf A$ corresponding to the two correlated values arranged in two columns.

Your problem is that $\Sigma^{-1/2}$ is not $\begin{pmatrix}1 & \frac{1}{\sqrt{\rho}} \\ \frac{1}{\sqrt{\rho}} & 1 \end{pmatrix}$, it is a matrix $T$ such that $$T \cdot T \cdot \Sigma = \Sigma \cdot T \cdot T = \begin{pmatrix}1&0\\0&1\end{pmatrix},$$ where $\cdot$ denotes matrix multiplication.

One possible such $T$ is (credit to Wolfram Alpha):

$$T := \frac{1}{2 \sqrt{1-\rho^2}}\begin{pmatrix}\sqrt{1-\rho}+\sqrt{1+\rho} & \sqrt{1-\rho}-\sqrt{1+\rho} \\ \sqrt{1-\rho}-\sqrt{1+\rho} & \sqrt{1-\rho}+\sqrt{1+\rho}\end{pmatrix}.$$

Observe that $$T^2 = \frac{1}{1-\rho^2}\begin{pmatrix}1 & -\rho \\ -\rho & 1 \end{pmatrix},$$ which is the inverse of your $\Sigma$.

• How would $T\,\begin{pmatrix}\bf X\\\bf Y\end{pmatrix}$ decorrelate $X$ from $Y$? – Antoni Parellada Apr 25 '16 at 3:47
• $\text{cov}(T\begin{pmatrix}X\\Y\end{pmatrix}, T\begin{pmatrix}X\\Y\end{pmatrix}) = T \cdot \text{cov}(\begin{pmatrix}X\\Y\end{pmatrix},\begin{pmatrix}X\\Y\end{pmatrix}) \cdot T^{\top} = T \cdot \Sigma \cdot T^{\top} = \begin{pmatrix}1&0\\0&1\end{pmatrix}$. – Anon Apr 25 '16 at 9:12
• I am accepting your answer, and thank you. However can you add a couple of lines explaining why $T\Sigma T^T=\Sigma TT=TT\Sigma$? – Antoni Parellada Apr 25 '16 at 10:46
• This is only the case because $T$ is symmetrical (any symmetrical positive semidefinite matrix has a symmetrical root, and covariance matrices are conveniently positive semidefinite): $T\Sigma T^\top = T\Sigma T = T^{-1} T T \Sigma T = T^{-1} I T = I = \Sigma T T = T T \Sigma$. – Anon Apr 25 '16 at 16:08

For a start:

$$\begin{bmatrix}1 & \rho\\\rho & 1\end{bmatrix}^{-1}~=~\dfrac1{1-\rho^2}\begin{bmatrix}-1 & \rho\\\rho & -1\end{bmatrix}$$

So

$$\begin{bmatrix}1 & \rho\\\rho & 1\end{bmatrix}^{-1/2}~=~\dfrac1{2}\begin{bmatrix}\dfrac 1{\sqrt{1+\rho}}+\dfrac 1{\sqrt{1-\rho}} & \dfrac 1{\sqrt{1+\rho}}-\dfrac 1{\sqrt{1-\rho}}\\\dfrac 1{\sqrt{1+\rho}}-\dfrac 1{\sqrt{1-\rho}} & \dfrac 1{\sqrt{1+\rho}}+\dfrac 1{\sqrt{1-\rho}}\end{bmatrix}$$

• Wouldn't $\frac{1}{\sqrt{1+p}}-\frac{1}{\sqrt{1+p}}$ in the matrix above just be $0$ – Antoni Parellada Apr 25 '16 at 3:34
• typo : obviously one of those plusses was mean to be a minus. – Graham Kemp Apr 25 '16 at 6:00

I did not read the answer you linked to, but here is what I think. Since the covariance is a bilinear form, under a change of basis $$\begin{pmatrix} X' \\ Y' \end{pmatrix} = \begin{pmatrix} a & b \\ c & d \end{pmatrix} \begin{pmatrix} X \\ Y \end{pmatrix}$$the covariance matrix will transform as $$\Sigma' = S \; \Sigma \; S^T$$ where $S$ is the $a,b,c,d$ matrix. So you want to find the $S$ that will make $\Sigma'$ identity. It is not hard to read off the eigenvectors and eigenvalues of $\Sigma$, from which it follows that $$S= \frac{1}{\sqrt{2}} \begin{pmatrix} \frac{1}{\sqrt{1 + \rho}} & \frac{1}{\sqrt{1 + \rho}} \\ -\frac{1}{\sqrt{1 - \rho}} & \frac{1}{\sqrt{1 - \rho}} \end{pmatrix}$$ I hope this will help!

• In the OP it seems as though $S$ multiplies the original matrix of correlated draws from the bivariate standard normal distribution to get a different matrix containing uncorrelated standard normal draws. Where does this matrix, $\begin{bmatrix}X\\Y\end{bmatrix}$ fits in the $S\Sigma S^T$ decomposition? – Antoni Parellada Apr 25 '16 at 1:05
• Do the following exercise. Suppose that you have $U = a X + b Y$ and $V = c X + d Y$, in other words $\begin{bmatrix}U \\ V\end{bmatrix} = \begin{bmatrix}a & b\\ c & d \end{bmatrix} \begin{bmatrix}X\\Y\end{bmatrix}$. Now calculated the correlation matrix for $U$ and $V$; this will prove the formula for the transformation of the covariance matrix I gave above. I am not sure I understand your question, but I think this might answer it anyway. – user226970 Apr 25 '16 at 1:22
• No, they are related by a matrix $S$. Sorry if it is unclear, I edited this. – user226970 Apr 25 '16 at 1:40