Why can we decompose Hermitian Matrices? In my quantum physics class today, I learned about Hermitian Matrices decomposition. The process can be described as follows:
Decomposing Hermitian Matrices:

*

*Find eigenvalues.

*Find eigenvectors.

*Find generator matrices using eigenvectors (where a generator matrix is defined as the outer product of an eigenvector with itself).

*Find the decomposition using eigenvalues and generator matrices.

I would like to know what relationship between eigenvalues and generator matrices allows such a decomposition. Namely, why does the relationship
$$
[M] = \sum\limits_{i=1}^n \lambda_i\ {| i\rangle}  {\langle i|}
$$
hold?
note:
$[M] =$ the Hermitian matrix; $\lambda_i =$ eigenvalue;
${| i\rangle}  {\langle i|} =$ generator
 A: The crucial relationship between what you call generator matrices and the eigenvalues is that the generator matrices are orthogonal projections (namely $|i\rangle\langle i|$ projects onto the linear subspace spanned by $|i\rangle$) and $M$ acts as $\lambda_i$ times the identity on this subspace.
Once you know that every Hermitian matrix has at least one eigenvalue, this is enough to show this decomposition as follows. Let $\lambda_1$ be an eigenvalue of $M$ with corresponding normalized eigenvector $|1\rangle$. Knowing that $|1\rangle\langle 1|$ is the orthogonal projection onto the linear subspace spanned by $|1\rangle$, we can write
$$
M|=M(|1\rangle\langle 1|+I-|1\rangle\langle 1|)=\lambda_1|1\rangle\langle 1|+M(I-|1\rangle\langle 1|).
$$
Now $I-|1\rangle\langle 1|$ is the orthogonal projection onto the orthogonal complement of $|1\rangle$. Since $M$ is hermitian, this space is also invariant under $M$, or in other words, $M(I-|1\rangle\langle 1|)=(I-|1\rangle\langle 1|)M(I-|1\rangle\langle 1|)=:M_1$.
The matrix $M_1$ is again hermitian on the $(n-1)$-dimensional subspace $|1\rangle^\perp$. This means you can play the same game again: Find and eigenvalue $\lambda_2$ with corresponding normalized eigenvector $|2\rangle$ and write $M_1$ as
$$
M_1=\lambda_2|2\rangle\langle 2|+M_1(I-|2\rangle\langle 2|).
$$
Thus
$$
M=\lambda_1|1\rangle\langle 1|+M_1=\lambda_1|1\rangle\langle 1|+\lambda_2|2\rangle\langle 2|+M_1(I-|2\rangle\langle 2|).
$$
Note that in every step we reduce the dimension by $1$, so that after $n$ steps (if $M$ is an $n\times n$ matrix) we arrive at
$$
M=\sum_{i=1}^n \lambda_i|i\rangle\langle i|.
$$
A: In general, if you have any basis $\mathbf v_1, \dots, \mathbf v_n$ of $\mathbb R^n$, you can define a linear transformation $f : \mathbb R^n \to \mathbb R^n$ simply by specifying $f(\mathbf v_1), \dots, f(\mathbf v_n)$.

If $\mathbf v_1, \dots, \mathbf v_n$ is an orthonormal basis, then $f$ has the matrix representation $f(\mathbf x) = M\mathbf x$ where
$$
   M = f(\mathbf v_1)\mathbf v_1^{\mathsf T} + \dots + f(\mathbf v_n) \mathbf v_n^{\mathsf T}.
$$
To check that this works, we can just take
$$M \mathbf v_i = f(\mathbf v_1)\mathbf v_1^{\mathsf T}\mathbf v_i + \dots + f(\mathbf v_i)\mathbf v_i^{\mathsf T}\mathbf v_i + \dots f(\mathbf v_n) \mathbf v_n^{\mathsf T}\mathbf v_i$$
and simplify: the term $f(\mathbf v_i)\mathbf v_i^{\mathsf T}\mathbf v_i$ simplifies to $f(\mathbf v_i)$, and all other terms simplify to $0$.

The key property of a symmetric matrix $M$ is that it has an orthonormal basis of eigenvectors: there is an orthonormal basis $\mathbf v_1, \dots, \mathbf v_n$ of $\mathbb R^n$ such that $M \mathbf v_i = \lambda_i \mathbf v_i$ for $i=1, \dots, n$.
So if we define $f(\mathbf x) = M\mathbf x$, then $f(\mathbf v_i) = \lambda_i\mathbf v_i$ for $i=1, \dots, n$, and therefore we can write $M$ as
$$
   M = \sum_{i=1}^n f(\mathbf v_i) \mathbf v_i^{\mathsf T} = \sum_{i=1}^n \lambda_i \mathbf v_i \mathbf v_i^{\mathsf T}
$$
which is the decomposition you want.
If you replace $\mathbb R^n$ by $\mathbb C^n$, $\mathbf x^{\mathsf T}$ by the conjugate transpose, and "symmetric" by "Hermitian", all of this still holds.
