2
$\begingroup$

I read somewhere that if $\Sigma$ is a covariance matrix, one can obtain the decomposition $\Sigma = \Sigma^{1/2}(\Sigma^{1/2})^T$ from Cholesky Decompositon. However, I am confused how because Cholesky decomposition requires an upper and lower triangular matrix. How can the square root here for upper and lower triangular, or can it?

$\endgroup$
3
$\begingroup$

Since $\Sigma^{-1}$ is a positive definite matrix, there is a lower triangular, nonsingular matrix $C$ such that

$$C^T\Sigma^{-1}C = I$$

Hence, $$I = I^{-1} = (C^T\Sigma^{-1}C)^{-1} = C^{-1} \Sigma (C^T)^{-1} \\ \implies CC^T = CIC^T = CC^{-1} \Sigma (C^T)^{-1}C^T = \Sigma.$$

For example, for the $2 \times 2$ case,

$$\Sigma = \pmatrix{\sigma_1^2 & \rho\sigma_1 \sigma_2 \\ \rho \sigma_1 \sigma_2 & \sigma_2^2} \\ \Sigma^{-1} = \frac{1}{1 - \rho^2}\pmatrix{\sigma_1^{-2} & -\rho\sigma_1^{-1} \sigma_2^{-1} \\ -\rho \sigma_1^{-1} \sigma_2^{-1} & \sigma_2^{-2}}.$$

$$C = \pmatrix{ \sigma_1 & 0 \\ \rho \sigma_2 & \sigma_2\sqrt{1-\rho^2}} .$$

$\endgroup$
  • $\begingroup$ Thanks, can I ask how you get the first result of the existence of a lower triangular nonsingular $C$? Is it from the spectral theorem? $\endgroup$ – user321627 Dec 18 '16 at 3:51
  • $\begingroup$ See here showing the algorithm to obtain Cholesky decomposition for any symmetric, positive definite matrix. As you can see from the example it all works. $\endgroup$ – RRL Dec 18 '16 at 4:05
  • $\begingroup$ I see, so you are saying that: $C^T\Sigma^{-1}C = I \implies \Sigma^{-1} = CC^T$ which is the classic form of Cholesky. Thanks! $\endgroup$ – user321627 Dec 18 '16 at 4:08
  • $\begingroup$ This is what I recall, and that transformation is the key to many useful results. The square root symbol is a convention in this case. Not the literal square root $\Sigma = BB$. $\endgroup$ – RRL Dec 18 '16 at 4:11
  • $\begingroup$ From Wikipedia on matrix square root.: "The Cholesky factorization provides another particular example of square root, which should not be confused with the unique non-negative square root." $\endgroup$ – RRL Dec 18 '16 at 4:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.