Is there any intuitive relationship between $A A^{T}$ and $A^{T} A$?

I'm struggling to understand the intuition behind the following arguments. If anyone please can point me in the right direction (in particular, a book that discuss this sort of construction).

Let $S_1 = A A^{T}$, $S_2 = A^{T} A$, $\Pi_1$ the projection onto the range of $S_1$, $\Pi_2$ the projection onto the range of $S_2$. Define $\tilde{A} = \lim_{\epsilon \downarrow 0} (\epsilon I + S_1)^{-1} \Pi_1$, $T = A^{T} \tilde{A}$ then $TA = \Pi_2$, $AT = \Pi_1$.

Let $M = \begin{pmatrix} S_1 & 0 \\ 0 & I_d \end{pmatrix}$ a $2d \times 2d$ matrix, $\Sigma = (T, I - \Pi_2)$ a $d \times 2d$ matrix, then $\Sigma M \Sigma^T = I_d$, $A\Sigma M = S_1$.

I can prove every claim, it's just that I don't understand the intuition behind it at all. It seems to me that there is something to it that I can't quite put my finger on. I think the crux lies in the relationship between $S_1$ and $S_2$, but I might be wrong. This construction comes out of nowhere in a book that only uses it as an auxiliary result, and I can't find any reference to this sort of thing anywhere. It feels like it should be something general and applicable elsewhere.

• what is the "out of nowhere" book? – Will Jagy May 27 '17 at 2:23
• They have the same nonzero eigenvalues, which are the squares of the absolute values of the singular values of $A$. Their eigenvectors are respectively the right and left singular vectors of $A$. – Qiaochu Yuan May 27 '17 at 2:55
• This is similar to @QiaochuYuan's comment, but if $A=U\Sigma V^T$ is an SVD of $A$ then you can see immediately that $A^T A=V \Sigma^T \Sigma V^T$ and $AA^T=U\Sigma^T\Sigma U^T$. So the singular vectors $v_i$ are eigenvectors of $A^TA$, the singular vectors $u_i$ are eigenvectors of $AA^T$, and the nonzero eigenvalues are squares of the singular values of $A$. – littleO May 27 '17 at 20:26

Here's a nice relationship between the two matrices: all matrices $A$ have polar decompositions $$A = P_1U = UP_2$$ Where $P_i = \sqrt{S_i}$ and $U$ is an orthogonal matrix (In fact, if $A$ is invertible, then $U$ is uniquely determined to be the nearest orthogonal matrix to $A$). With this orthogonal matrix $U$, we have $$AA^T = U(A^TA)U^T$$