3
$\begingroup$

Look at this screenshot:

eigendecomposition of a cov matrix

Now, the original matrix is a precision/covariance matrix and therefore real symmetric, and that allows him to do those algebraic manipulations, but that's not my point.

Look at the rightmost side, and tell me if I'm wrong: it seems he writes the eigendecomposition as sum of matrices, given that we have a product of columns times rows inside the summation.

Am I grasping it right?

$\endgroup$
1
  • $\begingroup$ Indeed. On the r.h.s. you have the summation of an outer product, which is a matrix. $\endgroup$
    – james42
    Jun 1, 2017 at 15:16

2 Answers 2

3
$\begingroup$

Yes, the RHS is a matrix, and that really only makes sense, as the LHS is a matrix, so they must be compatible.

You can easily see how to get to the formula by using block matrix multiplication.

Namely, you can write $U$ as a block matrix $U = \begin{bmatrix}u_1 & u_2 & \dots & u_D\end{bmatrix}$, where $u_i$ are column eigenvectors. Then $U^T = \begin{bmatrix}u_1^T \\ u_2^T \\ \vdots \\ u_D^T\end{bmatrix}$ and you do the multiplication:

\begin{align} U\Lambda U^T&=\begin{bmatrix}u_1 & u_2 & \dots & u_D\end{bmatrix} \begin{bmatrix}\lambda_1^{-1} & 0 & \dots & 0\\0 & \lambda_2^{-1} & \dots & 0\\ \vdots &\vdots & \ddots & \vdots\\ 0&0&0&\lambda_D^{-1} \end{bmatrix} \begin{bmatrix}u_1^T \\ u_2^T \\ \vdots \\ u_D^T\end{bmatrix}\\ &= \begin{bmatrix}\lambda_1^{-1}u_1 & \lambda_2^{-1}u_2 & \dots & \lambda_D^{-1}u_D\end{bmatrix}\begin{bmatrix}u_1^T \\ u_2^T \\ \vdots \\ u_D^T\end{bmatrix}\\ &=\lambda_1^{-1}u_1u_1^T + \lambda_2^{-1}u_2u_2^T+\dots+\lambda_D^{-1}u_Du_D^T\\ &=\sum_{i=1}^D\frac1{\lambda_i}u_iu_i^T \end{align}

$\endgroup$
1
$\begingroup$

You are right as $u_iu_i' = U^i$, where the ${kl}$ entry of $U^i$ matrix is $u_{ik} u_{il}$. These notations are not that rare in econometric and in general statistical models literature. Especiaiily, in discussion of the positive definiteness and related issues of the co-variance matrix.

$\endgroup$
3
  • $\begingroup$ Thanks. Do any of the single matrices we are summing up have a specific significance? $\endgroup$
    – MadHatter
    Jun 2, 2017 at 9:57
  • $\begingroup$ From the top of my head, maybe in diagnostics analysis of outliers and leverage points it can be useful too look at a single matrix that is formed out of a specific observation $u_i$. $\endgroup$
    – V. Vancak
    Jun 2, 2017 at 10:30
  • $\begingroup$ Thanks again...! $\endgroup$
    – MadHatter
    Jun 2, 2017 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.