# Orthogonal eigenvectors in symmetrical matrices with repeated eigenvalues and diagonalization

Symmetrical matrices have orthogonal eigenvectors. However, there is the special case when eigenvalues are repeated. The ultimate scenario is that of the identity matrix. Professor Strang mentions here that "if an eigenvalue is repeated, then there is a whole plane of eigenvectors, and in that plane we can choose perpendicular ones"... a "real substantial freedom."

And he goes on to note that symmetrical matrices can be diagonalized as ${\bf A = Q\Lambda Q'}$. Although he does mention "... I also mean that there is a full set of them [eigenvectors]", it sounds (in the video) as though the ${\bf Q\Lambda Q'}$ is not necessarily jeopardized by the presence of repeat eigenvalues.

However, and in general for square matrices (not limiting ourselves to symmetrical), repeated eigenvalues can render the matrix non-diagonalizable as $\bf{A=S\Lambda S^{-1}}$.

How does all this come together into a question? The ${\bf A'A}$ matrix has many properties shared by positive semidefinite matrices. Among them is its being diagonalizable. Now its eigenvalues do not have to necessarily be distinct (real, yes; but not necessarily of algebraic multiplicity of $1$).

So...

1. Is it correct to say that symmetrical matrices have always orthogonal eigenvectors (or we can choose them so that they are), guaranteeing the ${\bf Q\Lambda Q'}$ decomposition, regardless of the possible presence of repeat eigenvalues?

2. If (1) is not true, are we then stuck with a caveat to the assertion that ${\bf A'A}$ matrices are diagonalizable? Can we say that ${\bf A'A}$ is diagonalizable as a blanket statement?

By the spectral theorem, given a symmetric matrix $A$, there exists an orthonormal basis of eigenvectors of $A$. So, yes, a symmetric matrix always has orthogonal eigenvectors.

For example, the identity matrix has its only (repeated) eigenvalue as $1$, yet there does exist an orthonormal basis of eigenvectors of the identity matrix, regardless of the repeated eigenvalue.

In summary, you're correct to say that you can always find a basis of orthogonal eigenvectors given a symmetric matrix, regardless of whether the eigenvectors correspond to repeated eigenvalues.

• I checked the spectral theorem, and it settles the question in a binary way of true or false. But it would be great to understand why if $A'A$ can have repeat eigenvalues, it still enjoys the attribute of having a full set of non-linearly dependent eigenvectors without exception. – Antoni Parellada Aug 28 '16 at 14:21
• @AntoniParellada I gave you the example of the identity matrix. Clearly it is symmetric, and any orthogonal basis of the vector space in consideration is a basis of eigenvectors of the identity. Furthermore, each eigenvector corresponds to the one, single, unique eigenvalue, $1$. Hence, repeated eigenvalue and orthogonal eigenvectors. In general, repeated eigenvalues have nothing to do with the independence of eigenvectors. Hope that helps. – Alex Ortiz Aug 28 '16 at 14:24
• Thank you for clarifying the lack of connection between repeated eigenvalues and independence of eigenvectors. Repeated eigenvalues do have a connection to problems diagonalizing a matrix, though. In the case of ${\bf I}$ the solution is clear, but can we approach the case of ${\bf A'A}$ with repeated eigenvalues from first principles, and without having to resort to ${\bf I}$? – Antoni Parellada Aug 28 '16 at 14:31
• @AntoniParellada If your notation $A'$ means the transpose of the matrix $A$, then $A'A$ is always a symmetric matrix. Hence, it is orthogonally diagonalizable. The spectral theorem guarantees this orthogonal diagonalizability, regardless of repeated eigenvalues. – Alex Ortiz Aug 28 '16 at 14:36

General discussion: The nasty thing that may happen and prevent diagonalization is 'nil-potence'. Suppose $\lambda$ is an eigenvalue of $A$. Then $A-\lambda$ is neither injective nor surjective so $Z={\rm ker} (A-\lambda)$ is non-trivial and $W={\rm im} (A-\lambda)$ is not the full space. On the other hand we know that $\dim Z+\dim W=n$ (dimension of the full space). So if $Z\cap W=\{0\}$ (called transversality) all is fine, since $A-\lambda$ must now be a bijection of $W$ onto itself. We may then continue the game restricting the action to $W$ (which is invariant) and if for every eigenvalue we have transversality then all is fine and we have in fact diagonalized $A$ (agreed, ... in a somewhat abstract way and modulo a few details). In each eigenspace we are perfectly free to choose orthogonal eigenvectors.

The catch is that at some point $Z\cap W$ may be non-trivial which brings havoc to the argument and prevents diagonalization. This is precisely what happens with the matrix $$\left( \begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix} \right)$$ for which $Z=W={\rm Span} \{e_1\}$.

Concerning a symmetric matrix this never happens simply because $Z$ and $W$ are orthogonal! (the best transversality that we can ask for). To see this let $(A-\lambda)x=0$ and $y=(A-\lambda)v \neq 0$. Then $$\langle y,x \rangle = \langle Av,x\rangle = \langle v, Ax \rangle=0$$ So each eigenspace also ends up being orthogonal to each other and we may thus choose an orthogonal base of eigenvectors (I omit the discussion of the fact that all eigenvalues are real in this case).