Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to know, since the covariance matrix is symmetric, positive, and semi-definite, then if I calculate its eigenvectors what would be the properties of the space constructed by those eigenvectors (corresponds to non-close-zero eigenvalues), is it orthogonal or anything else special? Suppose this eigenvector matrix is called U, then what would be the properties with U*transpose(U)?

share|cite|improve this question
One can always set things up such that the matrix of eigenvectors of a symmetric positive semidefinite matrix is an orthogonal matrix, zero eigenvalues or not. – J. M. Apr 16 '11 at 19:08
How to set things up? and what would be the U*transpose(U) – Rn2dy Apr 16 '11 at 19:18
Most eigenroutines would generate an orthogonal matrix of eigenvectors. Mathematica and MATLAB (due to how LAPACK routines are set up) do. Remember that multiplying an orthogonal matrix with its transpose gives an identity matrix. – J. M. Apr 16 '11 at 19:20
@J.M.: Do you know what the eigenvectors of a symmetric positive semidefinite matrix mean if the matrix is not necessarily a covariance matrix? – Mitch Apr 17 '11 at 4:17
@Mitch: I tend to think of those things geometrically, much like Qiaochu's answer here... or you had something else in mind? – J. M. Apr 17 '11 at 4:42

A symmetric matrix has orthogonal eigenvectors (irrespective of being positive definite - or zero eigenvalues). Hence, if we normalize the eigenvectors, U * transpose(U) = I

share|cite|improve this answer
This assumes, of course, that orthogonal bases have been selected for any degenerate eigenspaces. For instance, $(1,0)$ and $(\sqrt{1/2}, \sqrt{1/2})$ form a normalized eigenbasis for the 2 by 2 identity matrix but $U U^T \ne I$. – whuber Apr 17 '11 at 17:56
@whuber: You're right. – leonbloy Apr 17 '11 at 21:26

The eigenvectors correspond to the principal components and the eigenvalues correspond to the variance explained by the principal components.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.