# $AA^T=A^TA$ and othogonallity of eigenvectors

In his Unit 3 Exam Review, Strang gives the theorem that a matrix $A$ has orthogonal eigenvectors if and only if $AA^T=A^TA$, and then lists symmetric, antisymmetric and orthogonal matrices as meeting this condition.

However, I know that in the symmetric case it's the eigenspaces that are guaranteed to be orthogonal, not the eigenvectors.

Is it correct to assume that he meant to say eigenspaces and not eigenvectors?

EDIT

So -- I have a -- you should say "why?" and I'll at least answer why for case one, maybe case two, the checking the Eigen -- that the eigenvectors are perpendicular, I'll leave to, the -- to the book. But let's just realize what -- well, first I have to say, it -- it could happen, like for the identity matrix -- there's a symmetric matrix. Its eigenvalues are certainly all real, they're all one for the identity matrix. What about the eigenvectors? Well, for the identity, every vector is an eigenvector. So how can I say they're perpendicular? What I really mean is the -- they -- this word are should really be written can be chosen perpendicular. That is, if we have -- it's the usual case. If the eigenvalues are all different, then each eigenvalue has one line of eigenvectors and those lines are perpendicular here. But if an eigenvalue's repeated, then there's a whole plane of eigenvectors and all I'm saying is that in that plain, we can choose perpendicular ones. So that's why it's a can be chosen part, is -- this is in the case of a repeated eigenvalue where there's some real, substantial freedom. But the typical case is different eigenvalues, all real, one dimensional eigenvector space, Eigen spaces, and all perpendicular.

-

I'm assuming Strang is allowing complex eigenvectors. The statement "$A$ has orthogonal eigenvectors" is a bit imprecise; a better statement would be "You can choose a basis of pairwise orthogonal eigenvectors". If you take his statement literally then it is correct but trivial, since a large enough normal matrix ($AA^*=A^* A$) will have some pair of orthogonal eigenvectors.

The eigenspaces for distinct eigenvalues are orthogonal, but that's different from saying there exists an orthonormal basis consisting of eigenvectors. The eigenspaces might not sum to the whole space.

-
So it's safe to assume that he meant "all the eigenspaces are othogonal", not "all the eigenvectors are orthogonal"? –  Robert S. Barnes Nov 2 '12 at 8:17
@RobertS.Barnes: The theorem you want to remember is "$A$ is normal matrix $\Leftrightarrow$ there is an orthonormal basis consisting of eigenvectors". This is equivalent to saying "All eigenspaces are orthogonal and the eigenspaces sum to the whole space". –  wj32 Nov 2 '12 at 8:22
OK, I guess I'll have to look up what a normal matrix is... –  Robert S. Barnes Nov 2 '12 at 8:27
@RobertS.Barnes: Normal matrix means $AA^*=A^* A$. For real matrices, normal just means $AA^T=A^T A$. –  wj32 Nov 2 '12 at 8:28
@RobertS.Barnes: That's right. Sometimes called "conjugate transpose", "Hermitian transpose", "adjoint". –  wj32 Nov 2 '12 at 8:35

For normal matrices, the eigenvectors corresponding to the same eigenvalue are linear independent, not necessary orthogonal. However, since any linear combination of two eigenvectors with the same eigenvalue is still an eigenvector associated with that same eigenvalue (i.e. the eigenvectors are not unique), we can always construct orthogonal eigenvectors through Gram-Schmidt procedure. In other words, if a normal matrix has eigenvalues with multiplicity, there are infinite sets of eigenvectors, some of which are orthogonal.

-

Whenever you have an eigenspace of dimension greater than $1$, you can of course find non-orthogonal bases of that space if you want to; there is nothing a matrix could undertake to prevent that (except not having such eigenspaces). However you can choose an orthonormal basis in the eigenspace. If you do that for every eigenspace and those eigenspaces are orthogonal, then you will have an orthonormal basis for their direct sum, which in this case is the whole space.

-

I assume all the matrices below are real. For a symmetric matrix, the eigenvectors are orthogonal. The proof is relatively straight forward. Let $A$ be the symmetric matrix. Consider two distinct eigenvalues say $\lambda$ and $\mu$. Let $x$ and $y$ be the associated eigenvectors.

Hence, we have that $$Ax = \lambda x; \,\,\,\, Ay = \mu y$$ We hence have that $$y^TAx = \lambda y^Tx; \,\,\,\, x^TAy = \mu x^T y$$ But note that $y^TAx = y^T(Ax) = (Ax)^T y= x^T A^T y = x^T Ay$. Hence, we get that $$\lambda y^Tx = \mu x^Ty$$ Again $y^Tx = x^Ty$ and since we chose $\lambda \neq \mu$. We have that $x^Ty = 0$. Hence, the eigenvectors corresponding to distinct eigenvalues are orthogonal.

One of the key properties of a symmetric matrix is that it is diagonalizable. Hence, if we have an eigenvalue with multiplicity $2$, there are two vectors $x$ and $y$ such that $Ax = \lambda x$ and $A y = \lambda y$, where $x \neq y$. But now any linear combination of $x$ and $y$ is also an eigenvector with the same eigenvalue. Hence, any vector in the space spanned by $x$ and $y$ is an eigenvector. Now choose any two orthogonal vectors in this space to be your eigenvectors.

-
But what about symmetric matrices where there is an eigenvalue of multiplicity greater than 1? IIRC, I've seen an example ( maybe in one of Strang's earlier lectures ) of such a matrix and the eigenvectors for said space eigenspace had to be perpendicularized ( if that's even a word ). Also, he made a point of saying in an earlier lecture that for a symmetric matrix the eigenvectors "can be chosen" orthogonal. –  Robert S. Barnes Nov 2 '12 at 8:14
@RobertS.Barnes I have updated my post to answer your question. –  user17762 Nov 2 '12 at 8:20