Lets say we have a symmetric positive semi-definite $n\times n$ matrix $\Sigma$, which therefore has a diagonalization $\Sigma=V\Lambda V^T$, where $V$ is an orthogonal matrix (containing the eigenvectors $v_1,...v_n$ of $\Sigma$) and $\Lambda$ is a diagonal matrix (containing the eigenvalues $\lambda_1,...,\lambda_n$ of $\Sigma$).

Could you show me how the following sum of outer products is derived?

$\Sigma=\sum_{i=1}^n \lambda_iv_iv_i^T$


It helps to have internalized the following ways of looking at matrix multiplication:

  • $Ax = a_1 x_1 + \cdots + a_n x_n$, where $a_i$ is the $i$th column of the matrix $A$ and $x_i$ is the $i$th component of the vector $x$.
  • $A \begin{bmatrix} b_1 & \cdots & b_n \end{bmatrix} = \begin{bmatrix} A b_1 & \cdots & A b_n \end{bmatrix}$, where $b_i$ is the $i$th column of the matrix $B$.
  • $\begin{bmatrix} a_1 & \cdots & a_n \end{bmatrix} \begin{bmatrix} b_1^T \\ \vdots \\ b_n^T \end{bmatrix} = a_1 b_1^T + \cdots + a_n b_n^T$. Here the vector $a_i$ is the $i$th column of $A$ and the row vector $b_i^T$ is the $i$th row of $B$.

You can use rules 2 and 1 to compute $V \Lambda$, and then use rule 3 to compute $V \Lambda V^T$.

Alternatively, you can think of it like this. The matrix $v_i v_i^T$ projects a vector $x$ onto the span of $\{ v_i \}$. This projection then gets scaled by $\lambda_i$. In other words: \begin{equation} x = \sum_{i=1}^n v_i v_i^T x \end{equation} so \begin{align} \Sigma x &= \sum_{i=1}^n \Sigma v_i v_i^T x \\ &= \sum_{i=1}^n \lambda_i v_i v_i^T x. \end{align}

  • $\begingroup$ Thank you very much! I've actually never thought of rule 3 - that one clarified everything for me. $\endgroup$ – ndrizza Jul 12 '16 at 6:59
  • $\begingroup$ Sure. Gilbert Strang emphasizes rule 3 (as well as the other viewpoints) in his linear algebra books. $\endgroup$ – littleO Jul 12 '16 at 7:05
  • $\begingroup$ Indeed his book is great. Unfortunately, I only only discovered it four days before my exam, so I only read selected chapters (which were great tough). I wish I had discovered his book before. $\endgroup$ – ndrizza Jul 12 '16 at 7:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.