Orthogonal eigenvectors in symmetrical matrices with repeated eigenvalues and diagonalization

Question

Symmetrical matrices have orthogonal eigenvectors. However, there is the special case when eigenvalues are repeated. The ultimate scenario is that of the identity matrix. Professor Strang mentions here that "if an eigenvalue is repeated, then there is a whole plane of eigenvectors, and in that plane we can choose perpendicular ones"... a "real substantial freedom."

And he goes on to note that symmetrical matrices can be diagonalized as ${\bf A = Q\Lambda Q'}$. Although he does mention "... I also mean that there is a full set of them [eigenvectors]", it sounds (in the video) as though the ${\bf Q\Lambda Q'}$ is not necessarily jeopardized by the presence of repeat eigenvalues.

However, and in general for square matrices (not limiting ourselves to symmetrical), repeated eigenvalues can render the matrix non-diagonalizable as $\bf{A=S\Lambda S^{-1}}$.

How does all this come together into a question? The ${\bf A'A}$ matrix has many properties shared by positive semidefinite matrices. Among them is its being diagonalizable. Now its eigenvalues do not have to necessarily be distinct (real, yes; but not necessarily of algebraic multiplicity of $1$).

So...

1. Is it correct to say that symmetrical matrices have always orthogonal eigenvectors (or we can choose them so that they are), guaranteeing the ${\bf Q\Lambda Q'}$ decomposition, regardless of the possible presence of repeat eigenvalues?

2. If (1) is not true, are we then stuck with a caveat to the assertion that ${\bf A'A}$ matrices are diagonalizable? Can we say that ${\bf A'A}$ is diagonalizable as a blanket statement?

Answer 1

By the spectral theorem, given a symmetric matrix $A$, there exists an orthonormal basis of eigenvectors of $A$. So, yes, a symmetric matrix always has orthogonal eigenvectors.

For example, the identity matrix has its only (repeated) eigenvalue as $1$, yet there does exist an orthonormal basis of eigenvectors of the identity matrix, regardless of the repeated eigenvalue.

In summary, you're correct to say that you can always find a basis of orthogonal eigenvectors given a symmetric matrix, regardless of whether the eigenvectors correspond to repeated eigenvalues.

Answer 2

General discussion: The nasty thing that may happen and prevent diagonalization is 'nil-potence'. Suppose $\lambda$ is an eigenvalue of $A$. Then $A-\lambda$ is neither injective nor surjective so $Z={\rm ker} (A-\lambda)$ is non-trivial and $W={\rm im} (A-\lambda)$ is not the full space. On the other hand we know that $\dim Z+\dim W=n$ (dimension of the full space). So if $Z\cap W=\{0\}$ (called transversality) all is fine, since $A-\lambda$ must now be a bijection of $W$ onto itself. We may then continue the game restricting the action to $W$ (which is invariant) and if for every eigenvalue we have transversality then all is fine and we have in fact diagonalized $A$ (agreed, ... in a somewhat abstract way and modulo a few details). In each eigenspace we are perfectly free to choose orthogonal eigenvectors.

The catch is that at some point $Z\cap W$ may be non-trivial which brings havoc to the argument and prevents diagonalization. This is precisely what happens with the matrix $$ \left( \begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix} \right)$$ for which $Z=W={\rm Span} \{e_1\}$.

Concerning a symmetric matrix this never happens simply because $Z$ and $W$ are orthogonal! (the best transversality that we can ask for). To see this let $(A-\lambda)x=0$ and $y=(A-\lambda)v \neq 0$. Then $$ \langle y,x \rangle = \langle Av,x\rangle = \langle v, Ax \rangle=0$$ So each eigenspace also ends up being orthogonal to each other and we may thus choose an orthogonal base of eigenvectors (I omit the discussion of the fact that all eigenvalues are real in this case).