# PCA - variance maximization

As has been stated in this thread:

It's not hard to show that if the covariance matrix of the original data points $x_i$ was $\Sigma$, the variance of the new data points is just $u^T \Sigma u$.

However, below it has been proven that the variance isn't equal to $\Sigma$, because we haven't divided $A$ by the number of points. Then $A$ is obviously a different matrix than the covariance matrix, thus it has different eigenvectors and eigenvalues than covariance matrix.

Most sources say we should take eigenvectors of $\Sigma$ as the new basis, but here we're taking eigenvectors of $A$. I'm a bit confused...

• Dividing by the number of points is just a scaling of $\mathbf{A}$ by a scalar. The eigenvectors will not change (the eigenvalues will -- they will all be scaled by the same number; the number of of points). – megas Mar 21 '15 at 18:59
• So if I have matrix $A$ and $\alpha A$, then both have the same eigenvectors but the eigenvalues of the latter are multiplied by $\alpha$? It sounds very basic, is it some property of eigenvalues and eigenvectors? Or a theorem that states that? – user4205580 Mar 21 '15 at 19:02
• Yes, they will have the same eigenvectors: $\mathbf{A}$ above is symmetric (in fact PSD) and can be written as $\mathbf{A}=\mathbf{U}\mathbf{\Lambda}\mathbf{U}^{T}$, where the columns of $\mathbf{U}$ are the (orthonormal) eigenvectors of $\mathbf{A}$ and $\mathbf{\Lambda}$ is a diagonal matrix containing the eigenvalues. Then, $\alpha \mathbf{A}= \alpha \mathbf{U}\mathbf{\Lambda}\mathbf{U}^{T}= \mathbf{U}\left( \alpha \mathbf{\Lambda}\right)\mathbf{U}^{T}$. – megas Mar 21 '15 at 19:04
• Thanks. How do we know that $\Lambda$ contains the eigenvalues? – user4205580 Mar 21 '15 at 20:55
• $\mathbf{\Lambda}$ was defined to contain the eigenvalues. If you have a symmetric matrix $\mathbf{A}$ you can always write/decompose it like that. – megas Mar 21 '15 at 23:37