Relationship between eigenvectors of matrices

Question

I am investigating parameter estimation in reduced-rank regression and have come across the following linear algebra result which I haven`t been able to prove. Suppose, $A \in \mathbb{R}^{nxm}$ of full rank, $B \in \mathbb{R}^{mxm}$ symmetric positive definte and $C \in \mathbb{R}^{nxn}$ also symmetric positive definite. Consider the following general eigenvalue-eigenvector problem:

$(ABA^T)U=CUD$

where

• $U \in \mathbb{R}^{nxm}$ is the matrix such that its columns are the eigenvectors of the general eigenvalue-eigenvector problem
• $D \in \mathbb{R}^{mxm}$ is the diagonal matrix such that its entries are the eigenvalues of the general eigenvalue-eigenvector problem

Now let:

• $(U_1,D_1)$ be the solution to the problem when $B=E^{-1}$ and,
• $(U_2,D_2)$ be the solution to the problem when $B=(E-A^TC^{-1}A)^{-1}$,
• with $E \in \mathbb{R}^{mxm}$, symmetric positive definite

We know that $U_1^TU_1=U_2^TU_2=I_m$. We also know that $U_1 \neq U_2$.

I want to prove that $U_1U_1^T=U_2U_2^T$. ¿Any pointers? Thanks!

Latest: The general eigenvalue-eigenvector problem above is equivalent to the following regular eigenvalue-eigenvector problem:

$(C^{-0.5}(ABA^T)C^{-0.5})X = XD$

where

• $X \in \mathbb{R}^{nxm}$ such that $U=C^{-0.5}X$.

Now if we set $S=B^{0.5}A^TC^{-0.5}$ we have that the eigenvalue-eigenvector problem is:

$(S^TS)X=XD$

where

• $S \in \mathbb{R}^{mxn}$
• $S$ has the following singular value decomposition: $S=YD^{0.5}X^T$
• $X$ has the eigenvectors of $S^TS$ and $Y$ has the eigenvectors of $SS^T$
• $D$ has the eigen values of both $S^TS$ and $SS^T$

By considering the eigenvalues of $SS^T$ instead of those of $S^TS$ I have been able to deduce a relationship between $D_1$ and $D_2$:

$SS^T=YDY^T=B^{0.5}A^TC^{-1}AB^{0.5}$

To obtain the eigenvalues of $SS^T$ we have to solve the following general eigenvalue characteristic equation:

$| \lambda B^{-1} - A^TC^{-1}A | = 0$

If we substitute the two possible values that $B$ can take we get the following two equations:

1. $|\lambda_1 E - A^TC^{-1}A|=0$
2. $|\lambda_2 (E-A^TC^{-1}A) - A^TC^{-1}A|=(1+ \lambda_2)^m|(\frac{\lambda_2}{1+ \lambda_2}) (E-A^TC^{-1}A) - A^TC^{-1}A|=0$

From this we can establish that:

• $\lambda_1 = \frac{\lambda_2}{1+ \lambda_2}$, and in general
• $D_1 = D_2(I_m + D_2)^{-1}$

However, I still haven't been able to deduce a relationship between $U_1$ and $U_2$ or equivalently between $X_1$ and $X_2$ that allows me to prove that $U_1U_1^T$=$U_2U_2^T$.

Answer 1

May be I missed something but it seems for me that you compare two matrices $X_1$ and $X_2$ $$ G_1 X_1=C^{-0.5}(AB_1A^T)C^{-0.5} X_1=X_1D_1 $$ and $$ G_2X_2=C^{-0.5}(AB_2A^T)C^{-0.5} X_2=X_2D_2 $$ It is easy to prove that $G_2,G_1$ has the same kernel and then $X_1,X_2$ are bases in the same space, Indeed $B_2=(E-A^TC^{−1}A)^{−1}$. By Woodberry indentity: $$ B_2=(E-A^TC^{−1}A)^{−1}=E^{-1}-E^{-1}A(C+A^TE^{-1}A)^{-1}A^TE^{-1} $$ So if $B_1A^TC^{-0.5}x=E^{-1}A^TC^{-0.5}x=0$ then $B_2A^TC^{-0.5}x=0$; From symmetry we can prove the opposite that if if $B_2A^TC^{-0.5}x=0$ then $B_1A^TC^{-0.5}x=0$. So $X_1$ $X_2$ are orthohormal vectors in the same space, thus $X_1X_1^T=X_2X_2^T$.
More precisely: $$ B_1A^TC^{-0.5}x=0 \Leftrightarrow B_2A^TC^{-0.5}x=0 \Leftrightarrow X_1^Tx=0 \Leftrightarrow X_2^Tx=0 $$

So let $R$ be a $nxn-m$ matrix of orthogonal vectors such that $$ B_1A^TC^{-0.5}R=B_2A^TC^{-0.5}R=0; $$ Then $$ X_1X_1^T=X_2X_2^T=I-RR^T $$