Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
Look at this screenshot:
Now, the original matrix is a precision/covariance matrix and therefore real symmetric, and that allows him to do those algebraic manipulations, but that's not my point.
Look at the rightmost side, and tell me if I'm wrong: it seems he writes the eigendecomposition as sum of matrices, given that we have a product of columns times rows inside the summation.
Am I grasping it right?
Yes, the RHS is a matrix, and that really only makes sense, as the LHS is a matrix, so they must be compatible.
You can easily see how to get to the formula by using block matrix multiplication.
Namely, you can write $U$ as a block matrix $U = \begin{bmatrix}u_1 & u_2 & \dots & u_D\end{bmatrix}$, where $u_i$ are column eigenvectors. Then $U^T = \begin{bmatrix}u_1^T \\ u_2^T \\ \vdots \\ u_D^T\end{bmatrix}$ and you do the multiplication:
\begin{align} U\Lambda U^T&=\begin{bmatrix}u_1 & u_2 & \dots & u_D\end{bmatrix} \begin{bmatrix}\lambda_1^{-1} & 0 & \dots & 0\\0 & \lambda_2^{-1} & \dots & 0\\ \vdots &\vdots & \ddots & \vdots\\ 0&0&0&\lambda_D^{-1} \end{bmatrix} \begin{bmatrix}u_1^T \\ u_2^T \\ \vdots \\ u_D^T\end{bmatrix}\\ &= \begin{bmatrix}\lambda_1^{-1}u_1 & \lambda_2^{-1}u_2 & \dots & \lambda_D^{-1}u_D\end{bmatrix}\begin{bmatrix}u_1^T \\ u_2^T \\ \vdots \\ u_D^T\end{bmatrix}\\ &=\lambda_1^{-1}u_1u_1^T + \lambda_2^{-1}u_2u_2^T+\dots+\lambda_D^{-1}u_Du_D^T\\ &=\sum_{i=1}^D\frac1{\lambda_i}u_iu_i^T \end{align}
You are right as $u_iu_i' = U^i$, where the ${kl}$ entry of $U^i$ matrix is $u_{ik} u_{il}$. These notations are not that rare in econometric and in general statistical models literature. Especiaiily, in discussion of the positive definiteness and related issues of the co-variance matrix.
