I understand how the covariance matrix can be used to find the orientation of a data cloud. For example, in 2-D, for zero mean data, the direction of the major axis of the cloud of data is given by the eigenvector associated with the largest eigenvalue.
Let $A$ be an $m\times n$ data matrix with $m>n=2$. Define the covariance matrix $C=A^TA$. Then, the eigenvectors of $C$ will give me the principle direction of the data cloud. So far so good.
I've also seen it suggested that if $A$ and $B$ are $m\times n$ data matrices, then, the SVD of $C'=A^TB$ will give me the rotation from $A$ to $B$. If $C'=U\Sigma V^T$, then $R=VU^T$ where $R$ is the desired rotation matrix.
However, I'm having a hard time understanding the latter result. I know that the singular values of $C'$ are the square roots of the eigenvalues of $C'^2=C'^TC'$ but I don't want to use $C'^2$. I'd prefer it if I could just use the eigenvalues and eigenvectors of $C'$ just as I did with $C$. But, I can't convince myself that this will give me the right result. For example, I tried setting $B=AR$ to see what effect the rotation matrix would have on the eigenvalues. I was hoping they would be the same but, unless I made some mistake, $\lambda'\ne\lambda$.
Am I doing something wrong? Is there some way to manually calculate the SVD of a 2x2 covariance (or cross-covariance) matrix other than by squaring it?
Any help would be greatly appreciated.