when we do the SVD for a $m\times n$ matrix, we have to extend the set $u_1, ... , u_r$, to an orthonormal basis $u_1, ... , u_m$ for $R^m$ if $r<m$. But why don't we just fill zero vectors to make everything easier?

for example: enter image description here why not: enter image description here

as we actually don't need to use components in $u_3$?

  • $\begingroup$ You want $U$ and $V$ to be orthogonal matrices. This requires a full orthonormal basis for the columns. $\endgroup$ – EuYu Jun 8 '16 at 10:04

There are at least two reasons why you shouldn't have an all zero column.

First, while the decomposition might be numerically the same, having $U$ and $V$ invertible, and more over orthogonal, is very desirable. You seem to interpret the decomposition as being simpler with an all zero column. This is not the case. Invertible/orthogonal matrices are much easier to handle, both theoretically and numerically, than singular matrices. Having $U$ and $V$ be orthogonal at the expensive of carrying out Gram-Schmidt for a couple of more iterations seems like a small price to pay.

Secondly, and this is related to the first point, is that the singular value decomposition is not just a numerical decomposition. It is a representation of the linear map $\Sigma$ with respect to orthonormal basis of its domain and codomain. The matrices $U$ and $V$ reflect a choice of basis (or rather rotation to a specific basis), and that's the reason they're orthogonal matrices in the first place. You lose this interpretation when you stop with an all zero column, and this is not very desirable.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.