1
$\begingroup$

I am investigating parameter estimation in reduced-rank regression and have come across the following linear algebra result which I haven`t been able to prove. Suppose, $A \in \mathbb{R}^{nxm}$ of full rank, $B \in \mathbb{R}^{mxm}$ symmetric positive definte and $C \in \mathbb{R}^{nxn}$ also symmetric positive definite. Consider the following general eigenvalue-eigenvector problem:

$(ABA^T)U=CUD$

where

  • $U \in \mathbb{R}^{nxm}$ is the matrix such that its columns are the eigenvectors of the general eigenvalue-eigenvector problem
  • $D \in \mathbb{R}^{mxm}$ is the diagonal matrix such that its entries are the eigenvalues of the general eigenvalue-eigenvector problem

Now let:

  • $(U_1,D_1)$ be the solution to the problem when $B=E^{-1}$ and,
  • $(U_2,D_2)$ be the solution to the problem when $B=(E-A^TC^{-1}A)^{-1}$,
  • with $E \in \mathbb{R}^{mxm}$, symmetric positive definite

We know that $U_1^TU_1=U_2^TU_2=I_m$. We also know that $U_1 \neq U_2$.

I want to prove that $U_1U_1^T=U_2U_2^T$. ¿Any pointers? Thanks!

Latest: The general eigenvalue-eigenvector problem above is equivalent to the following regular eigenvalue-eigenvector problem:

$(C^{-0.5}(ABA^T)C^{-0.5})X = XD$

where

  • $X \in \mathbb{R}^{nxm}$ such that $U=C^{-0.5}X$.

Now if we set $S=B^{0.5}A^TC^{-0.5}$ we have that the eigenvalue-eigenvector problem is:

$(S^TS)X=XD$

where

  • $S \in \mathbb{R}^{mxn}$
  • $S$ has the following singular value decomposition: $S=YD^{0.5}X^T$
  • $X$ has the eigenvectors of $S^TS$ and $Y$ has the eigenvectors of $SS^T$
  • $D$ has the eigen values of both $S^TS$ and $SS^T$

By considering the eigenvalues of $SS^T$ instead of those of $S^TS$ I have been able to deduce a relationship between $D_1$ and $D_2$:

$SS^T=YDY^T=B^{0.5}A^TC^{-1}AB^{0.5}$

To obtain the eigenvalues of $SS^T$ we have to solve the following general eigenvalue characteristic equation:

$| \lambda B^{-1} - A^TC^{-1}A | = 0$

If we substitute the two possible values that $B$ can take we get the following two equations:

  1. $|\lambda_1 E - A^TC^{-1}A|=0$
  2. $|\lambda_2 (E-A^TC^{-1}A) - A^TC^{-1}A|=(1+ \lambda_2)^m|(\frac{\lambda_2}{1+ \lambda_2}) (E-A^TC^{-1}A) - A^TC^{-1}A|=0$

From this we can establish that:

  • $\lambda_1 = \frac{\lambda_2}{1+ \lambda_2}$, and in general
  • $D_1 = D_2(I_m + D_2)^{-1}$

However, I still haven't been able to deduce a relationship between $U_1$ and $U_2$ or equivalently between $X_1$ and $X_2$ that allows me to prove that $U_1U_1^T$=$U_2U_2^T$.

$\endgroup$
  • $\begingroup$ I think this should be a fairly straightforward SVD problem... $\endgroup$ – Ian Jul 18 '14 at 17:44
  • $\begingroup$ Thanks @Ian. I have had some joy using SVD in relating $D_1$ with $D_2$, however I still haven't figured out how to use this to relate $U_1$ with $U_2$. Any ideas? $\endgroup$ – emDiaz Jul 18 '14 at 19:12
2
$\begingroup$

May be I missed something but it seems for me that you compare two matrices $X_1$ and $X_2$ $$ G_1 X_1=C^{-0.5}(AB_1A^T)C^{-0.5} X_1=X_1D_1 $$ and $$ G_2X_2=C^{-0.5}(AB_2A^T)C^{-0.5} X_2=X_2D_2 $$ It is easy to prove that $G_2,G_1$ has the same kernel and then $X_1,X_2$ are bases in the same space, Indeed $B_2=(E-A^TC^{−1}A)^{−1}$. By Woodberry indentity: $$ B_2=(E-A^TC^{−1}A)^{−1}=E^{-1}-E^{-1}A(C+A^TE^{-1}A)^{-1}A^TE^{-1} $$ So if $B_1A^TC^{-0.5}x=E^{-1}A^TC^{-0.5}x=0$ then $B_2A^TC^{-0.5}x=0$; From symmetry we can prove the opposite that if if $B_2A^TC^{-0.5}x=0$ then $B_1A^TC^{-0.5}x=0$. So $X_1$ $X_2$ are orthohormal vectors in the same space, thus $X_1X_1^T=X_2X_2^T$.
More precisely: $$ B_1A^TC^{-0.5}x=0 \Leftrightarrow B_2A^TC^{-0.5}x=0 \Leftrightarrow X_1^Tx=0 \Leftrightarrow X_2^Tx=0 $$

So let $R$ be a $nxn-m$ matrix of orthogonal vectors such that $$ B_1A^TC^{-0.5}R=B_2A^TC^{-0.5}R=0; $$ Then $$ X_1X_1^T=X_2X_2^T=I-RR^T $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.