Peculiar way of writing eigendecomposition of a real symmetric matrix.

Look at this screenshot:

Now, the original matrix is a precision/covariance matrix and therefore real symmetric, and that allows him to do those algebraic manipulations, but that's not my point.

Look at the rightmost side, and tell me if I'm wrong: it seems he writes the eigendecomposition as sum of matrices, given that we have a product of columns times rows inside the summation.

Am I grasping it right?

• Indeed. On the r.h.s. you have the summation of an outer product, which is a matrix. Jun 1, 2017 at 15:16

Yes, the RHS is a matrix, and that really only makes sense, as the LHS is a matrix, so they must be compatible.

You can easily see how to get to the formula by using block matrix multiplication.

Namely, you can write $U$ as a block matrix $U = \begin{bmatrix}u_1 & u_2 & \dots & u_D\end{bmatrix}$, where $u_i$ are column eigenvectors. Then $U^T = \begin{bmatrix}u_1^T \\ u_2^T \\ \vdots \\ u_D^T\end{bmatrix}$ and you do the multiplication:

\begin{align} U\Lambda U^T&=\begin{bmatrix}u_1 & u_2 & \dots & u_D\end{bmatrix} \begin{bmatrix}\lambda_1^{-1} & 0 & \dots & 0\\0 & \lambda_2^{-1} & \dots & 0\\ \vdots &\vdots & \ddots & \vdots\\ 0&0&0&\lambda_D^{-1} \end{bmatrix} \begin{bmatrix}u_1^T \\ u_2^T \\ \vdots \\ u_D^T\end{bmatrix}\\ &= \begin{bmatrix}\lambda_1^{-1}u_1 & \lambda_2^{-1}u_2 & \dots & \lambda_D^{-1}u_D\end{bmatrix}\begin{bmatrix}u_1^T \\ u_2^T \\ \vdots \\ u_D^T\end{bmatrix}\\ &=\lambda_1^{-1}u_1u_1^T + \lambda_2^{-1}u_2u_2^T+\dots+\lambda_D^{-1}u_Du_D^T\\ &=\sum_{i=1}^D\frac1{\lambda_i}u_iu_i^T \end{align}

You are right as $u_iu_i' = U^i$, where the ${kl}$ entry of $U^i$ matrix is $u_{ik} u_{il}$. These notations are not that rare in econometric and in general statistical models literature. Especiaiily, in discussion of the positive definiteness and related issues of the co-variance matrix.

• Thanks. Do any of the single matrices we are summing up have a specific significance? Jun 2, 2017 at 9:57
• From the top of my head, maybe in diagnostics analysis of outliers and leverage points it can be useful too look at a single matrix that is formed out of a specific observation $u_i$. Jun 2, 2017 at 10:30
• Thanks again...! Jun 2, 2017 at 14:17