Orthogonality of the degenerate eigenvectors of a real symmetric matrix

Question

It is relatively easy to show for a real symmetric matrix $ A $ that its eigenvectors belonging to distinct eigenvalues are orthogonal; it comes down to $(\lambda_i - \lambda_j) u_i^Tu_j=0$ and since eigenvalues are different; the eigenvectors have to be orthogonal. When the eigenvalues are equal, I know that we can pick eigenvectors which are orthogonal to eachother and to all other eigenvectors, enabling to build an orthogonal basis of eigenvectors which span $\mathbb {R^N} $ for a $ N \times N $ matrix. I try to show how one can pick orthogonal vectors for a shared eigenvalue $\lambda $. I tried to use the characteristic polynomial $ det (A - \lambda I_N)=0$ which has multiple roots at a shared eigenvalue $\lambda $. Assuming that $\lambda $ has multiplicity of $ m $ I tried to show then the matrix $ A - \lambda I_N $ has an $ m $ dimensional nullspace, spanned by $ m $ eigenvectors, but failed to reach any conclusions. How can we construct a proof of that?

Answer 1

The typical approach to this problem is not to show directly that an eigenvalue with multiplicity $m$ for a symmetric matrix has an $m$-dimensional space of corresponding eigenvectors but to use an inductive argument which shows it indirectly. In order to do that, it is more comfortable to talk about self-adjoint maps instead of real symmetric matrices (because the induction is done by letting the matrix act on an invariant subspace which doesn't correspond neatly to a submatrix or something like that).

Given a self-adjoint linear map $T \colon \mathbb{R}^n \rightarrow \mathbb{R}^n$ you can show three things:

1. If $W \subseteq \mathbb{R}^n$ is $T$-invariant subspace ($T(W) \subseteq W$) then $W^{\perp}$ is also $T$-invariant. This is done by a straightforward calculation.
2. If $W \subseteq \mathbb{R}^n$ is $T$-invariant then $T|_W \colon W \rightarrow W$ is also self-adjoint with respect to the inner-product on $W$ obtained by restricting the standard inner-product on $\mathbb{R}^n$.
3. The map $T$ has a eigenvector - that is, there exists $0 \neq v \in \mathbb{R}^n$ such that $T(v) = \lambda v$ for some $\lambda \in \mathbb{R}$. This is quite delicate. If $T(v) = Av$ for a symmetric $A \in M_n(\mathbb{R})$ then one way to see it is to let $A$ act on $\mathbb{C}^n$ and consider $S \colon \mathbb{C}^n \rightarrow \mathbb{C}^n$ given by $S(v) = Av$. The map $S$ is still self-adjoint and so has real eigenvalues. Since we are working over the complex numbers, the map $S$ must have a complex eigenvector $w \in \mathbb{C}^n$ with $Sw = Aw = \lambda w$ for $\lambda \in \mathbb{R}$. But $\mathrm{rank}_{\mathbb{C}} |\lambda I - A| = \mathrm{rank}_{\mathbb{R}} |\lambda I - A|$ there must also be a non-zero real vector $v \in \mathbb{R}^n$ with $Av = \lambda v$.

Assuming you have shown the items above, you can show that $T$ must be diagonalizable by an inductive argument. Start with an eigenvector $0 \neq v \in \mathbb{R}^n$ for $T$ and let $W := \mathrm{span} \{ v \}$. Then $W$ is $T$-invariant and so $W^{\perp}$ is $T$-invariant and $T|_{W^{\perp}} \colon W^{\perp} \rightarrow W^{\perp}$ is self-adjoint. Repeat the argument for $T|_{W^{\perp}}$. In the end you'll obtain an orthogonal basis $(v_1, \ldots, v_n)$ of eigenvectors for $T$ which shows that $T$ is diagonalizable and indirectly shows that the algebraic multiplicity of each eigenvalue must coincide with the geometric multiplicity.

Answer 2

This seems to address your problem, I found a very nice basis for the eigenvectors of a matrix with all entries $1.$ The reason we know the columns are independent is that they are perpendicular to each other, ordinary dot product of columns is zero. I am encouraging you to do something along these lines.

$$ \left( \begin{array}{rrrrrrrrrr} 1 & -1 & -1 & -1 & -1 & -1 & -1 & -1 & -1 & -1 \\ 1 & 1 & -1 & -1 & -1 & -1 & -1 & -1 & -1 & -1 \\ 1 & 0 & 2 & -1 & -1 & -1 & -1 & -1 & -1 & -1 \\ 1 & 0 & 0 & 3 & -1 & -1 & -1 & -1 & -1 & -1 \\ 1 & 0 & 0 & 0 & 4 & -1 & -1 & -1 & -1 & -1 \\ 1 & 0 & 0 & 0 & 0 & 5 & -1 & -1 & -1 & -1 \\ 1 & 0 & 0 & 0 & 0 & 0 & 6 & -1 & -1 & -1 \\ 1 & 0 & 0 & 0 & 0 & 0 & 0 & 7 & -1 & -1 \\ 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 8 & -1 \\ 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 9 \end{array} \right). $$

The columns of $P$ are of varying lengths; for the 10 by 10 case depicted, lengths $ \sqrt{10}, \sqrt{2}, \sqrt{6}, \sqrt{12},..$ All that is necessary to make an orthogonal matrix $Q$ out of this is to divide each column by its length.

Answer 3

All eigenvectors sharing a common eigenvalue $\lambda $ form a subspace: the kernel of the operator $A-\lambda I$. Using the Gram-Schmidt process we can construct an orthonormal basis for this subspace. The vectors of this basis have the needed properties.