Eigenvectors of real symmetric matrices are orthogonal

Question

Can someone point me to a paper, or show here, why symmetric matrices have orthogonal eigenvectors? In particular, I'd like to see proof that for a symmetric matrix $A$ there exists decomposition $A = Q\Lambda Q^{-1} = Q\Lambda Q^{T}$ where $\Lambda$ is diagonal.

Answer 1

For any real matrix $A$ and any vectors $\mathbf{x}$ and $\mathbf{y}$, we have $$\langle A\mathbf{x},\mathbf{y}\rangle = \langle\mathbf{x},A^T\mathbf{y}\rangle.$$ Now assume that $A$ is symmetric, and $\mathbf{x}$ and $\mathbf{y}$ are eigenvectors of $A$ corresponding to distinct eigenvalues $\lambda$ and $\mu$. Then $$\lambda\langle\mathbf{x},\mathbf{y}\rangle = \langle\lambda\mathbf{x},\mathbf{y}\rangle = \langle A\mathbf{x},\mathbf{y}\rangle = \langle\mathbf{x},A^T\mathbf{y}\rangle = \langle\mathbf{x},A\mathbf{y}\rangle = \langle\mathbf{x},\mu\mathbf{y}\rangle = \mu\langle\mathbf{x},\mathbf{y}\rangle.$$ Therefore, $(\lambda-\mu)\langle\mathbf{x},\mathbf{y}\rangle = 0$. Since $\lambda-\mu\neq 0$, then $\langle\mathbf{x},\mathbf{y}\rangle = 0$, i.e., $\mathbf{x}\perp\mathbf{y}$.

Now find an orthonormal basis for each eigenspace; since the eigenspaces are mutually orthogonal, these vectors together give an orthonormal subset of $\mathbb{R}^n$. Finally, since symmetric matrices are diagonalizable, this set will be a basis (just count dimensions). The result you want now follows.

Answer 2

It would appear that you want to write vectors as rows, so your preferred multiplication will be on the left side, as in $v \mapsto v A.$

The ordinary dot product is then $ v \cdot w = v w^T = w v^T = w \cdot v.$ Note that $v w^T$ is a number, or a 1 by 1 matrix, and is equal to its transpose.

In the same way, $v A \cdot w = v A w^T.$ However, $v A w^T$ is again a 1 by 1 matrix and is equal to its transpose, and $A^T = A,$ so we get $$ v A \cdot w = v A w^T = (v A w^T)^T = (w^T)^T A^T v^T = w A v^T = w A \cdot v$$

First suppose $v,w$ are eigenvectors with distinct eigenvalues $\lambda, \mu.$ We have $$ v A \cdot w = \lambda v \cdot w = w A \cdot v = \mu w \cdot v.$$ Or, $\lambda v \cdot w = \mu v \cdot w,$ finally $$ (\lambda - \mu) v \cdot w = 0.$$ So, eigenvectors with distinct eigenvalues are orthogonal.

It is possible that an eigenvalue may have larger multiplicity. However, for a fixed eigenvalue $\lambda,$ the set of vectors $v$ for which $ v A = \lambda v$ is a subspace, of full dimension (meaning the Jacobi form has no off-diagonal elements), and we may simply choose an orthonormal basis for this subspace. Choosing, in this way, all basis vectors to be length 1 and orthogonal, we get an orthonormal basis of eigenvalues of $A.$ Write those as rows of a matrix $P,$ we get $P A P^T = \Lambda.$

The only difficult aspect here is this: if an eigenvalue has algebraic multiplicity larger than one, that is the characteristic polynmial has a factor of $(x-\lambda)^k$ for some $k \geq 2,$ how can I be sure that the geometric multiplicity is also $k?$ That is, with $A$ symmetric, how do I know that $$ v (A - \lambda I)^k = 0 \; \; \Rightarrow \; \; v (A - \lambda I) = 0?$$ Working on it. It appears that this is, at heart, induction on $k,$ and takes many pages. Give me some time.

Alright, this works. An induction on dimension shows that every matrix is orthogonal similar to an upper triangular matrix, with the eigenvalues on the diagonal (the precise statement is unitary similar). How do we know the eigenvalues are real? We have an eigenvalue $\lambda$ with an eigenvector $v,$ perhaps both with complex entries. As is traditional, for a vector or matrix define $v^\ast = \bar{v}^T$ and $A^\ast = \bar{A}^T.$ It is easy to see that $v v^\ast$ is a positive real number unless $v = 0.$ In any case $A^\ast = A.$ So, given $v A = \lambda v,$ $$ ( v A v^\ast)^\ast = (v^\ast)^\ast A^\ast v^\ast = v A v^\ast.$$ As a result, the complex number $v A v^\ast$ is actually a real number. At the same time, $v A v^\ast = \lambda v v^\ast,$ and since both $v A v^\ast$ and $v v^\ast$ are real numbers, the latter nonzero, it follows that $\lambda$ is real.

Put these together, we get that each real matrix with real characteristic values is orthogonal similar to an upper triangular real matrix. However, as $A$ is symmetric, this upper triangular matrix is actually diagonal.