How To Prove Left Singular Vectors Are Orthogonal?

Question

For SVD, for a matrix $A$ that is $m~\times~n$, we select the right singular vectors as

$$ \mathbf{v_1} \equiv \text{arg max}_{\mathbf{v}:\vert\mathbf{v}\vert=1}\vert{A\mathbf{v}}\vert $$

$$ \mathbf{v_i} \equiv \text{arg max}_{\mathbf{v}\perp\mathbf{v_1},...,\mathbf{v}\perp\mathbf{v_{i-1}},\vert\mathbf{v}\vert=1}\vert{A\mathbf{v}}\vert $$

And we select the singular values and left singular vectors as $$ \sigma_{1}\equiv\text{max}_{\mathbf{v}:\vert\mathbf{v}\vert=1}\vert A\mathbf{v}\vert $$ $$ \sigma_{i}\equiv\text{max}_{\mathbf{v}\perp\mathbf{v_1},...,\mathbf{v}\perp\mathbf{v_{i-1}},\vert\mathbf{v}\vert=1}\vert A\mathbf{v}\vert $$ $$ \mathbf{u_i}\equiv\frac{A\mathbf{v_i}}{\sigma_{i}} $$ How can I prove that the $\mathbf{u_i}$'s, the left singular vectors, are orthogonal? I know of the proof by induction, but I was wondering if there is a simpler proof?

I have read this answer: https://math.stackexchange.com/a/1838857/515257

But that answer assumes that $A^T\mathbf{u_i}=\sigma_i\mathbf{v_i}$. But I'm not making that assumption. Or is there a way to derive that?

Answer 1

It can be derived using the Gramian matrix and Gram-Schmidt Orthonormalization Process, and the Gram-Schmidt Orthonormalization Process ensures that the singular vectors are orthonormal and hence orthogonal. For more information please refer to this tutorial: Singular Value Decomposition Tutorial.