0
$\begingroup$

I was reading this answer, and the first sentence seemed more intuitive at first than after thinking through it:

If $\pmatrix{X\\ Y}$ is bivariate normal with mean $\pmatrix{0\\0}$ and covariance matrix $\Sigma=\pmatrix{1&\rho\\\rho&1}$, then $\pmatrix{U\\V}=\Sigma^{-1/2} \pmatrix{X\\Y}$ is bivariate normal with mean $\pmatrix{0\\0}$ and covariance matrix $\pmatrix{1&0\\ 0&1}.$ That is, $U$ and $V$ are independent, standard normal random variables.

At first I thought that $\Sigma^{-1/2}= \begin{pmatrix}1&&\frac{1}{\sqrt{\rho}}\\\frac{1}{\sqrt{\rho}}&&1\end{pmatrix}$.

But this is clearly not the case as the answers so far explain.

This mistake corrected, I still would like to understand why:

$\frac{1}{2 \sqrt{1-\rho^2}}\begin{pmatrix}\sqrt{1-\rho}+\sqrt{1+\rho} & \sqrt{1-\rho}-\sqrt{1+\rho} \\ \sqrt{1-\rho}-\sqrt{1+\rho} & \sqrt{1-\rho}+\sqrt{1+\rho}\end{pmatrix}\begin{pmatrix}\mathbf X\\ \mathbf Y\end{pmatrix}$ manages to de-correlate $\bf X$ and $\bf Y$.

Or, $\frac{1}{2 \sqrt{1-\rho^2}}\begin{pmatrix}\sqrt{1-\rho}+\sqrt{1+\rho} & \sqrt{1-\rho}-\sqrt{1+\rho} \\ \sqrt{1-\rho}-\sqrt{1+\rho} & \sqrt{1-\rho}+\sqrt{1+\rho}\end{pmatrix} \mathbf A^T$ with $\mathbf A$ corresponding to the two correlated values arranged in two columns.

$\endgroup$
1
$\begingroup$

Your problem is that $\Sigma^{-1/2}$ is not $\begin{pmatrix}1 & \frac{1}{\sqrt{\rho}} \\ \frac{1}{\sqrt{\rho}} & 1 \end{pmatrix}$, it is a matrix $T$ such that $$T \cdot T \cdot \Sigma = \Sigma \cdot T \cdot T = \begin{pmatrix}1&0\\0&1\end{pmatrix},$$ where $\cdot$ denotes matrix multiplication.

One possible such $T$ is (credit to Wolfram Alpha):

$$T := \frac{1}{2 \sqrt{1-\rho^2}}\begin{pmatrix}\sqrt{1-\rho}+\sqrt{1+\rho} & \sqrt{1-\rho}-\sqrt{1+\rho} \\ \sqrt{1-\rho}-\sqrt{1+\rho} & \sqrt{1-\rho}+\sqrt{1+\rho}\end{pmatrix}.$$

Observe that $$T^2 = \frac{1}{1-\rho^2}\begin{pmatrix}1 & -\rho \\ -\rho & 1 \end{pmatrix},$$ which is the inverse of your $\Sigma$.

$\endgroup$
  • $\begingroup$ How would $T\,\begin{pmatrix}\bf X\\\bf Y\end{pmatrix}$ decorrelate $X$ from $Y$? $\endgroup$ – Antoni Parellada Apr 25 '16 at 3:47
  • $\begingroup$ $\text{cov}(T\begin{pmatrix}X\\Y\end{pmatrix}, T\begin{pmatrix}X\\Y\end{pmatrix}) = T \cdot \text{cov}(\begin{pmatrix}X\\Y\end{pmatrix},\begin{pmatrix}X\\Y\end{pmatrix}) \cdot T^{\top} = T \cdot \Sigma \cdot T^{\top} = \begin{pmatrix}1&0\\0&1\end{pmatrix}$. $\endgroup$ – Anon Apr 25 '16 at 9:12
  • $\begingroup$ I am accepting your answer, and thank you. However can you add a couple of lines explaining why $T\Sigma T^T=\Sigma TT=TT\Sigma$? $\endgroup$ – Antoni Parellada Apr 25 '16 at 10:46
  • 1
    $\begingroup$ This is only the case because $T$ is symmetrical (any symmetrical positive semidefinite matrix has a symmetrical root, and covariance matrices are conveniently positive semidefinite): $T\Sigma T^\top = T\Sigma T = T^{-1} T T \Sigma T = T^{-1} I T = I = \Sigma T T = T T \Sigma$. $\endgroup$ – Anon Apr 25 '16 at 16:08
1
$\begingroup$

For a start:

$$\begin{bmatrix}1 & \rho\\\rho & 1\end{bmatrix}^{-1}~=~\dfrac1{1-\rho^2}\begin{bmatrix}-1 & \rho\\\rho & -1\end{bmatrix}$$

So

$$\begin{bmatrix}1 & \rho\\\rho & 1\end{bmatrix}^{-1/2}~=~\dfrac1{2}\begin{bmatrix}\dfrac 1{\sqrt{1+\rho}}+\dfrac 1{\sqrt{1-\rho}} & \dfrac 1{\sqrt{1+\rho}}-\dfrac 1{\sqrt{1-\rho}}\\\dfrac 1{\sqrt{1+\rho}}-\dfrac 1{\sqrt{1-\rho}} & \dfrac 1{\sqrt{1+\rho}}+\dfrac 1{\sqrt{1-\rho}}\end{bmatrix}$$

$\endgroup$
  • 1
    $\begingroup$ Wouldn't $\frac{1}{\sqrt{1+p}}-\frac{1}{\sqrt{1+p}}$ in the matrix above just be $0$ $\endgroup$ – Antoni Parellada Apr 25 '16 at 3:34
  • $\begingroup$ typo : obviously one of those plusses was mean to be a minus. $\endgroup$ – Graham Kemp Apr 25 '16 at 6:00
1
$\begingroup$

I did not read the answer you linked to, but here is what I think. Since the covariance is a bilinear form, under a change of basis $$ \begin{pmatrix} X' \\ Y' \end{pmatrix} = \begin{pmatrix} a & b \\ c & d \end{pmatrix} \begin{pmatrix} X \\ Y \end{pmatrix} $$the covariance matrix will transform as $$ \Sigma' = S \; \Sigma \; S^T $$ where $ S $ is the $ a,b,c,d $ matrix. So you want to find the $ S $ that will make $ \Sigma' $ identity. It is not hard to read off the eigenvectors and eigenvalues of $ \Sigma $, from which it follows that $$ S= \frac{1}{\sqrt{2}} \begin{pmatrix} \frac{1}{\sqrt{1 + \rho}} & \frac{1}{\sqrt{1 + \rho}} \\ -\frac{1}{\sqrt{1 - \rho}} & \frac{1}{\sqrt{1 - \rho}} \end{pmatrix} $$ I hope this will help!

$\endgroup$
  • $\begingroup$ In the OP it seems as though $S$ multiplies the original matrix of correlated draws from the bivariate standard normal distribution to get a different matrix containing uncorrelated standard normal draws. Where does this matrix, $\begin{bmatrix}X\\Y\end{bmatrix}$ fits in the $S\Sigma S^T$ decomposition? $\endgroup$ – Antoni Parellada Apr 25 '16 at 1:05
  • $\begingroup$ Do the following exercise. Suppose that you have $ U = a X + b Y $ and $ V = c X + d Y $, in other words $ \begin{bmatrix}U \\ V\end{bmatrix} = \begin{bmatrix}a & b\\ c & d \end{bmatrix} \begin{bmatrix}X\\Y\end{bmatrix} $. Now calculated the correlation matrix for $ U $ and $ V $; this will prove the formula for the transformation of the covariance matrix I gave above. I am not sure I understand your question, but I think this might answer it anyway. $\endgroup$ – user226970 Apr 25 '16 at 1:22
  • $\begingroup$ No, they are related by a matrix $ S $. Sorry if it is unclear, I edited this. $\endgroup$ – user226970 Apr 25 '16 at 1:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.