Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I have a very simple question that can be stated without proof. Are all eigenvectors, of any matrix, always orthogonal? I am trying to understand Principal components and it is cruucial for me to see the basis of eigenvectors.

share|improve this question
3  
No. Take any any non-orthogonal basis $(v_1,\dots,v_n)$ and define a linear map $A$ on this basis by sending each $v_i$ to $iv_i$. The eigenspaces are the $n$ lines generated by the $v_i$, and these are by construction not ortgogonal. –  Olivier Bégassat May 8 '12 at 13:38
5  
Eigenvectors corresponding to different eigenvalues will be orthogonal if the matrix is symmetric. This is part of the real spectral theorem. –  Dylan Moreland May 8 '12 at 13:38
    
The case @Dylan is describing will apply to your study of principal components, since the underlying matrices are symmetric... –  J. M. May 8 '12 at 14:50

1 Answer 1

up vote 5 down vote accepted

Fix two linearly independent vectors $u$ and $v$ in $\mathbb{R}^2$, define $Tu=u$ and $Tv=2v$. Then extend linearly $T$ to a map from $\mathbb{R}^n$ to itself. The eigenvectors of $T$ are $u$ and $v$ (or any multiple). Of course, $u$ need not be perpendicular to $v$.

share|improve this answer
    
Thank you very much for your answers. Could anyone state whether they are orthogonal in PCA case? –  Bober02 May 8 '12 at 13:45
    
For PCA, things can always be set up such that the eigenvectors are orthogonal. On the other hand, I would recommend looking at PCA as a singular value decomposition instead of as an eigendecomposition. It's been discussed here on math.SE a number of times; search around. –  J. M. May 8 '12 at 14:48
    
What do you mean by "setting up"? Is there some common technique to achive a singular matrix? –  Bober02 May 9 '12 at 8:12
    
My understanding based on en.wikipedia.org/wiki/Principal_component_analysis is that a singular value decomposition (SVD) of $X$ results in three matrices, where the first of them is composed of the eigenvectors of $X^T X$, and is used for PCA. This matrix is always orthogonal. –  Uri Jun 14 '13 at 13:04
    
Beautiful answer. –  Don Larynx Nov 13 '13 at 15:14

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.