For principal component analysis we usually use correlation matrix when the data are measured in different units. In case where data units are consistent we can use covariance matrix instead.

I want to ask if there is any known mathematical relationship between eigenvalues of the correlation matrix and the covariance matrix, so that if we know one of them, we might be able to compute the other by some formula or algorithm simpler than running the entire PCA again?

  • 1
    $\begingroup$ What is the difference between "covariance matrix" and "correlation matrix"? If they only differ by a nonzero scalar multiple, then eigenvalues of one are scalar multiples of the other. That is, if $r\neq 0$, then $\lambda$ is an eigenvalue of $A$ if and only if $r\lambda$ is an eigenvalue of $rA$. $\endgroup$ – Michael Jun 30 '17 at 1:20
  • 1
    $\begingroup$ Thanks for comment. I am not sure, but I have trouble understanding why "covariance matrix" $\Sigma$ is a scalar multiple of correlation matrix: ${\rm{corr}}({\bf{X}}) = {\left( {{\rm{diag}}(\Sigma )} \right)^{ - \frac{1}{2}}}{\mkern 1mu} \Sigma {\mkern 1mu} {\left( {{\rm{diag}}(\Sigma )} \right)^{ - \frac{1}{2}}}$ $\endgroup$ – Tony Jun 30 '17 at 11:12
  • $\begingroup$ So then you are normalizing each component of your random vector $(X_1, ..., X_n)$ by its individual standard deviation (rather than normalizing the whole vector by some scalar). The new vector is then $(X_1/\sigma_1, ..., X_n/\sigma_n)$. Then I agree your diag matrix is not necessarily a scalar multiple of the identity, and the resulting covariance matrix of the new vector is not necessarily a scalar multiple of the old (it would be a scalar multiple if all standard deviations were the same). $\endgroup$ – Michael Jun 30 '17 at 19:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.