# Relationship between eigenvectors of matrices

I am investigating parameter estimation in reduced-rank regression and have come across the following linear algebra result which I haven`t been able to prove. Suppose, $A \in \mathbb{R}^{nxm}$ of full rank, $B \in \mathbb{R}^{mxm}$ symmetric positive definte and $C \in \mathbb{R}^{nxn}$ also symmetric positive definite. Consider the following general eigenvalue-eigenvector problem:

$(ABA^T)U=CUD$

where

• $U \in \mathbb{R}^{nxm}$ is the matrix such that its columns are the eigenvectors of the general eigenvalue-eigenvector problem
• $D \in \mathbb{R}^{mxm}$ is the diagonal matrix such that its entries are the eigenvalues of the general eigenvalue-eigenvector problem

Now let:

• $(U_1,D_1)$ be the solution to the problem when $B=E^{-1}$ and,
• $(U_2,D_2)$ be the solution to the problem when $B=(E-A^TC^{-1}A)^{-1}$,
• with $E \in \mathbb{R}^{mxm}$, symmetric positive definite

We know that $U_1^TU_1=U_2^TU_2=I_m$. We also know that $U_1 \neq U_2$.

I want to prove that $U_1U_1^T=U_2U_2^T$. ¿Any pointers? Thanks!

Latest: The general eigenvalue-eigenvector problem above is equivalent to the following regular eigenvalue-eigenvector problem:

$(C^{-0.5}(ABA^T)C^{-0.5})X = XD$

where

• $X \in \mathbb{R}^{nxm}$ such that $U=C^{-0.5}X$.

Now if we set $S=B^{0.5}A^TC^{-0.5}$ we have that the eigenvalue-eigenvector problem is:

$(S^TS)X=XD$

where

• $S \in \mathbb{R}^{mxn}$
• $S$ has the following singular value decomposition: $S=YD^{0.5}X^T$
• $X$ has the eigenvectors of $S^TS$ and $Y$ has the eigenvectors of $SS^T$
• $D$ has the eigen values of both $S^TS$ and $SS^T$

By considering the eigenvalues of $SS^T$ instead of those of $S^TS$ I have been able to deduce a relationship between $D_1$ and $D_2$:

$SS^T=YDY^T=B^{0.5}A^TC^{-1}AB^{0.5}$

To obtain the eigenvalues of $SS^T$ we have to solve the following general eigenvalue characteristic equation:

$| \lambda B^{-1} - A^TC^{-1}A | = 0$

If we substitute the two possible values that $B$ can take we get the following two equations:

1. $|\lambda_1 E - A^TC^{-1}A|=0$
2. $|\lambda_2 (E-A^TC^{-1}A) - A^TC^{-1}A|=(1+ \lambda_2)^m|(\frac{\lambda_2}{1+ \lambda_2}) (E-A^TC^{-1}A) - A^TC^{-1}A|=0$

From this we can establish that:

• $\lambda_1 = \frac{\lambda_2}{1+ \lambda_2}$, and in general
• $D_1 = D_2(I_m + D_2)^{-1}$

However, I still haven't been able to deduce a relationship between $U_1$ and $U_2$ or equivalently between $X_1$ and $X_2$ that allows me to prove that $U_1U_1^T$=$U_2U_2^T$.

• I think this should be a fairly straightforward SVD problem...
– Ian
Jul 18, 2014 at 17:44
• Thanks @Ian. I have had some joy using SVD in relating $D_1$ with $D_2$, however I still haven't figured out how to use this to relate $U_1$ with $U_2$. Any ideas? Jul 18, 2014 at 19:12

May be I missed something but it seems for me that you compare two matrices $X_1$ and $X_2$ $$G_1 X_1=C^{-0.5}(AB_1A^T)C^{-0.5} X_1=X_1D_1$$ and $$G_2X_2=C^{-0.5}(AB_2A^T)C^{-0.5} X_2=X_2D_2$$ It is easy to prove that $G_2,G_1$ has the same kernel and then $X_1,X_2$ are bases in the same space, Indeed $B_2=(E-A^TC^{−1}A)^{−1}$. By Woodberry indentity: $$B_2=(E-A^TC^{−1}A)^{−1}=E^{-1}-E^{-1}A(C+A^TE^{-1}A)^{-1}A^TE^{-1}$$ So if $B_1A^TC^{-0.5}x=E^{-1}A^TC^{-0.5}x=0$ then $B_2A^TC^{-0.5}x=0$; From symmetry we can prove the opposite that if if $B_2A^TC^{-0.5}x=0$ then $B_1A^TC^{-0.5}x=0$. So $X_1$ $X_2$ are orthohormal vectors in the same space, thus $X_1X_1^T=X_2X_2^T$.
More precisely: $$B_1A^TC^{-0.5}x=0 \Leftrightarrow B_2A^TC^{-0.5}x=0 \Leftrightarrow X_1^Tx=0 \Leftrightarrow X_2^Tx=0$$
So let $R$ be a $nxn-m$ matrix of orthogonal vectors such that $$B_1A^TC^{-0.5}R=B_2A^TC^{-0.5}R=0;$$ Then $$X_1X_1^T=X_2X_2^T=I-RR^T$$