Eigenvalues and Eigenvectors of $X'X$ and $XX'$

I am trying to derive (or prove) the relationship between the eigenvalues and eigenvectors of the matrices $X'X$ and $XX'$. It is fairly intuitive that they are related but I cannot derive the relationship. The result is simply stated in passing as part of another proof in a multivariate stats methods book but when I tried to work it out I couldn't. The $X$ are sample vectors so real numbers, say dimension ($n \times p$). The result is:

If $l_k$ and $\mathbb a_k$ are the $k^{th}$ eigenvalue and eigenvector of $X'X$, then the $k^{th}$ eigenvalue and eigenvector of XX' are $l_k$ and ${l_k}^{-1/2}X\mathbb a_k$.

Can someone point me to a good reference explaining this or show me how?

Many thanks for the help.

-

Since $l_k^{-\frac{1}{2}}$ is not defined for $l_k=0$ I will assume $l_k\neq 0$.

Let $a_k\neq 0$ such that $YXa_k = l_k a_k$, then $$XY \left(l_k^{-\frac{1}{2}}Xa_k\right) = Xl_k^{-\frac{1}{2}}(YXa_k) = Xl_k^{-\frac{1}{2}}(l_k a_k) = l_k\left(l_k^{-\frac{1}{2}}Xa_k\right).$$

Notice that the factor $l_k^{-\frac{1}{2}}$ is actually superfluous as any nonzero multiple of an eigenvector is an eigenvector as well. Futhermore $Xa_k\neq 0$ since $Y(Xa_k) = l_k a_k\neq 0$.

-
Abel, thanks. This was very helpful. I did get this far myself but this didn't answer for me where the scaling came from. Your answer along with copper.hat helped me think it through further and I took a crack at what I think might be a less general but more complete answer to my question below. Apologies if answering my own question isn't great etiquette or appears ungrateful. –  Gordon Apr 27 '13 at 18:18

The underlying result is more general:

If $\lambda \neq 0$ is an eigenvalue of $AB$ with eigenvector $v$, then $ABv = \lambda v$. Hence $BABv = BA (Bv) = \lambda Bv$ (and $Bv \neq 0$, otherwise $ABv=0$).

Hence if $\lambda \neq 0$ is an eigenvalue of $AB$ with eigenvector $v$, then $\lambda$ is an eigenvalue of $BA$ with eigenvector $Bv$.

Take $A=X^T$, $B=X$.

Note that if $X=\begin{bmatrix} 1 \\ 0 \end{bmatrix}$, then $X^T X=1$ has exactly one eigenvalue at $1$, but $X X^T$ has eigenvalues $\{0,1\}$.

-
copper.hat, thanks. This was very helpful. I did get this far myself but this didn't answer for me where the scaling came from. Your answer along with Abel's helped me think it through further, and as noted above apologies if answering my own question isn't great etiquette or appears ungrateful. Thanks also for what looks like a corner case. I don't think I will run into this as I using this in a statistics context and X is big with many rows and columns. –  Gordon Apr 27 '13 at 18:20
You're welcome. In general, unless $X$ is square, one of the products $X^T X$ or $X X^T$ will have zero eigenvalues, so that it not a corner case. I'm nor sure what you mean by scaling. Any non-zero multiple of the eigenvector is an eigenvector. –  copper.hat Apr 27 '13 at 19:25