The SVD is usually written as $A = U\Lambda V^T$. However, V is an orthogonal matrix. So if we represent $V^T$ as $V^*$ (I write asterisk here to differ it from V before), but not the transpose one. It still suffices to the requirement. e.g ${V^*}^T V^*=I$. So why do we bother to write it as transpose version? What's the goodness of doing so? I think $A = U\Lambda V$ is all the same here.

It troubles me for sometimes and I don't find quite valuable info on this topic so far.

Thank you!

  • $\begingroup$ Singular value... $\endgroup$ – user251257 Dec 17 '17 at 19:31
  • 3
    $\begingroup$ It is written this way because it is conventional to have the singular vectors as the columns of $U$ and $V$. This retains the similarity with the orthogonal diagonalization of a symmetric matrix, $A=PDP^T$, where eigenvectors of $A$ form the columns of $P$. If we wrote the SVD the way you propose, the left singular vectors would be the columns of $V$, but the right singular vectors would be the rows of $V_*$. $\endgroup$ – Rahul Dec 17 '17 at 19:33
  • $\begingroup$ @Rahul Do you mean the columns of U in the last sentence? So it's the convention to think the rows of the righthand matrix are right singular vectors and thus we use transpose here to make it uniform with U? $\endgroup$ – Li haonan Dec 17 '17 at 19:49

If the (orthonormal) columns of $V$ are $v_1, \ldots, v_k$, then $x \mapsto V^t x$ expresses $x$ in terms of its coordinates in the $V$-basis. So writing it that way helps one see something about an interpretation of the decomposition.

Also: when the matrix $A$ is not square, the decomposition theorem (in slightly modified form) still applies, and it's still true that the columns of $U$ and the columns of $V$ are orthogonal unit vectors, but since the matrices no longer need be square, it's not necessarily true of the rows. Example: $$ V = \pmatrix {0 & 0 \\ 1 & 0 \\ 0 & 1 } $$ the columns are orthogonal unit vectors, but the rows are not.

Could the theorem say that $A$ can be written $$ U D W $$ where $U$ has orthonormal columns and $W$ has orthonormal rows? Sure. But the other formulation happens to carry suggestions about rewriting things in other bases that some of us find helpful. (Others, like you, may not.)

(And, as Rahul notes in a comment, this puts the singular vectors in columns; if you're used to mostly working with column vectors, that's nice.)

  • $\begingroup$ I think I got it here. Thank you for your illustration! $\endgroup$ – Li haonan Dec 17 '17 at 19:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.