I'm struggling to understand the intuition behind the following arguments. If anyone please can point me in the right direction (in particular, a book that discuss this sort of construction).

Let $S_1 = A A^{T}$, $S_2 = A^{T} A$, $\Pi_1$ the projection onto the range of $S_1$, $\Pi_2$ the projection onto the range of $S_2$. Define $\tilde{A} = \lim_{\epsilon \downarrow 0} (\epsilon I + S_1)^{-1} \Pi_1$, $T = A^{T} \tilde{A}$ then $TA = \Pi_2$, $AT = \Pi_1$.

Let $M = \begin{pmatrix} S_1 & 0 \\ 0 & I_d \end{pmatrix}$ a $2d \times 2d$ matrix, $\Sigma = (T, I - \Pi_2)$ a $d \times 2d$ matrix, then $\Sigma M \Sigma^T = I_d$, $A\Sigma M = S_1$.

I can prove every claim, it's just that I don't understand the intuition behind it at all. It seems to me that there is something to it that I can't quite put my finger on. I think the crux lies in the relationship between $S_1$ and $S_2$, but I might be wrong. This construction comes out of nowhere in a book that only uses it as an auxiliary result, and I can't find any reference to this sort of thing anywhere. It feels like it should be something general and applicable elsewhere.

  • 2
    $\begingroup$ what is the "out of nowhere" book? $\endgroup$ – Will Jagy May 27 '17 at 2:23
  • 1
    $\begingroup$ They have the same nonzero eigenvalues, which are the squares of the absolute values of the singular values of $A$. Their eigenvectors are respectively the right and left singular vectors of $A$. $\endgroup$ – Qiaochu Yuan May 27 '17 at 2:55
  • $\begingroup$ This is similar to @QiaochuYuan's comment, but if $A=U\Sigma V^T$ is an SVD of $A$ then you can see immediately that $A^T A=V \Sigma^T \Sigma V^T$ and $AA^T=U\Sigma^T\Sigma U^T$. So the singular vectors $v_i$ are eigenvectors of $A^TA$, the singular vectors $u_i$ are eigenvectors of $AA^T$, and the nonzero eigenvalues are squares of the singular values of $A$. $\endgroup$ – littleO May 27 '17 at 20:26

Here's a nice relationship between the two matrices: all matrices $A$ have polar decompositions $$ A = P_1U = UP_2 $$ Where $P_i = \sqrt{S_i}$ and $U$ is an orthogonal matrix (In fact, if $A$ is invertible, then $U$ is uniquely determined to be the nearest orthogonal matrix to $A$). With this orthogonal matrix $U$, we have $$ AA^T = U(A^TA)U^T $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.