For SVD, for a matrix $A$ that is $m~\times~n$, we select the right singular vectors as

$$ \mathbf{v_1} \equiv \text{arg max}_{\mathbf{v}:\vert\mathbf{v}\vert=1}\vert{A\mathbf{v}}\vert $$

$$ \mathbf{v_i} \equiv \text{arg max}_{\mathbf{v}\perp\mathbf{v_1},...,\mathbf{v}\perp\mathbf{v_{i-1}},\vert\mathbf{v}\vert=1}\vert{A\mathbf{v}}\vert $$

And we select the singular values and left singular vectors as $$ \sigma_{1}\equiv\text{max}_{\mathbf{v}:\vert\mathbf{v}\vert=1}\vert A\mathbf{v}\vert $$ $$ \sigma_{i}\equiv\text{max}_{\mathbf{v}\perp\mathbf{v_1},...,\mathbf{v}\perp\mathbf{v_{i-1}},\vert\mathbf{v}\vert=1}\vert A\mathbf{v}\vert $$ $$ \mathbf{u_i}\equiv\frac{A\mathbf{v_i}}{\sigma_{i}} $$ How can I prove that the $\mathbf{u_i}$'s, the left singular vectors, are orthogonal? I know of the proof by induction, but I was wondering if there is a simpler proof?

I have read this answer: https://math.stackexchange.com/a/1838857/515257

But that answer assumes that $A^T\mathbf{u_i}=\sigma_i\mathbf{v_i}$. But I'm not making that assumption. Or is there a way to derive that?


It can be derived using the Gramian matrix and Gram-Schmidt Orthonormalization Process, and the Gram-Schmidt Orthonormalization Process ensures that the singular vectors are orthonormal and hence orthogonal. For more information please refer to this tutorial: Singular Value Decomposition Tutorial.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.