SVD: Matrix size mismatch: Part 1 I'm reading this article.
In the "Proof" section, there is an equation:
$A^T = (USV^T)^T = VS^TU^T = VSU^T$
I don't understand the $VS^TU^T = VSU^T$ part.
Is this $VSU^T$ simply a typo?
If we borrow the following matrix A from the "Example" section:
$$
A=
\begin{pmatrix}
3 & 2 & 2\\\ 2 & 3 & -2
\end{pmatrix}
$$
, the shape of S would be 2 x 3:
$$
S=
\begin{pmatrix}
5 & 0 & 0\\\ 0 & 3 & 0
\end{pmatrix}
$$
So, $VS^TU^T$ works because (3 x 3)@(3 x 2)@(2 x 2),
while $VSU^T$ doesn't work because (3 x 3)@(2 x 3)@(2 x 2).
Am I correct, or missing something?
 A: It isn't a typo. It is a genuine mistake. $A^TA$ is equal to $VS^TSV^T$, not $VS^2V^T$. Since $S$ in general is not a square matrix, it cannot be squared in the first place.
In fact, not only is the equality $A^TA=VS^2V^T$ wrong, the author's attempt to prove the existence of SVD is also wrong.
In essence, the author tries to prove that if $A=USV^T$, then each column of $V$ is necessarily an eigenvector of $A^TA$ and each column of $U$ is necessarily an eigenvector of $AA^T$. However, he hasn't justifed why $U$ and $V$ exist. That is, he hasn't justified why we can always find an orthonormal set of eigenvectors of $A^TA$ or $AA^T$.
But there is an even bigger issue. The author tries to solve for $V$ and $U$ separately without considering the consistency between the two solutions. However, given an orthogonal matrix $V$ whose columns are eigenvectors of $A^TA$ and given an orthogonal matrix $U$ whose columns are eigenvectors of $AA^T$, it is not necessarily true that $S=U^TAV$ is a diagonal matrix with a nonnegative diagonal. For example, pick any orthogonal matrix $A\ne I$. Then $A^TA=AA^T=I$. If we take $U=V=S=I$, then $A^TAV=S^TSV$ and $AA^TU=SS^TU$. Yet, $A\ne I=USV^T$.
