I read somewhere that if $\Sigma$ is a covariance matrix, one can obtain the decomposition $\Sigma = \Sigma^{1/2}(\Sigma^{1/2})^T$ from Cholesky Decompositon. However, I am confused how because Cholesky decomposition requires an upper and lower triangular matrix. How can the square root here for upper and lower triangular, or can it?
2
$\begingroup$
$\endgroup$
3
$\begingroup$
$\endgroup$
Since $\Sigma^{-1}$ is a positive definite matrix, there is a lower triangular, nonsingular matrix $C$ such that
$$C^T\Sigma^{-1}C = I$$
Hence, $$I = I^{-1} = (C^T\Sigma^{-1}C)^{-1} = C^{-1} \Sigma (C^T)^{-1} \\ \implies CC^T = CIC^T = CC^{-1} \Sigma (C^T)^{-1}C^T = \Sigma.$$
For example, for the $2 \times 2$ case,
$$\Sigma = \pmatrix{\sigma_1^2 & \rho\sigma_1 \sigma_2 \\ \rho \sigma_1 \sigma_2 & \sigma_2^2} \\ \Sigma^{-1} = \frac{1}{1 - \rho^2}\pmatrix{\sigma_1^{-2} & -\rho\sigma_1^{-1} \sigma_2^{-1} \\ -\rho \sigma_1^{-1} \sigma_2^{-1} & \sigma_2^{-2}}.$$
$$C = \pmatrix{ \sigma_1 & 0 \\ \rho \sigma_2 & \sigma_2\sqrt{1-\rho^2}} .$$
-
$\begingroup$ Thanks, can I ask how you get the first result of the existence of a lower triangular nonsingular $C$? Is it from the spectral theorem? $\endgroup$ – user321627 Dec 18 '16 at 3:51
-
-
$\begingroup$ I see, so you are saying that: $C^T\Sigma^{-1}C = I \implies \Sigma^{-1} = CC^T$ which is the classic form of Cholesky. Thanks! $\endgroup$ – user321627 Dec 18 '16 at 4:08
-
$\begingroup$ This is what I recall, and that transformation is the key to many useful results. The square root symbol is a convention in this case. Not the literal square root $\Sigma = BB$. $\endgroup$ – RRL Dec 18 '16 at 4:11
-
$\begingroup$ From Wikipedia on matrix square root.: "The Cholesky factorization provides another particular example of square root, which should not be confused with the unique non-negative square root." $\endgroup$ – RRL Dec 18 '16 at 4:15
