I understand how the covariance matrix can be used to find the orientation of a data cloud. For example, in 2-D, for zero mean data, the direction of the major axis of the cloud of data is given by the eigenvector associated with the largest eigenvalue.

Let $A$ be an $m\times n$ data matrix with $m>n=2$. Define the covariance matrix $C=A^TA$. Then, the eigenvectors of $C$ will give me the principle direction of the data cloud. So far so good.

I've also seen it suggested that if $A$ and $B$ are $m\times n$ data matrices, then, the SVD of $C'=A^TB$ will give me the rotation from $A$ to $B$. If $C'=U\Sigma V^T$, then $R=VU^T$ where $R$ is the desired rotation matrix.

However, I'm having a hard time understanding the latter result. I know that the singular values of $C'$ are the square roots of the eigenvalues of $C'^2=C'^TC'$ but I don't want to use $C'^2$. I'd prefer it if I could just use the eigenvalues and eigenvectors of $C'$ just as I did with $C$. But, I can't convince myself that this will give me the right result. For example, I tried setting $B=AR$ to see what effect the rotation matrix would have on the eigenvalues. I was hoping they would be the same but, unless I made some mistake, $\lambda'\ne\lambda$.

Am I doing something wrong? Is there some way to manually calculate the SVD of a 2x2 covariance (or cross-covariance) matrix other than by squaring it?

Any help would be greatly appreciated.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.