Given a $n \times m$ real matrix $A$ of rank $r$ one can define its SVD as $A = UD V^T$ with $D$ being a $r \times r$ diagonal matrix and $U^TU = V^TV= I$. Here clearly the diagonal entries of $D$ are not guaranteed to be be all positive.

  • Hence in terms of the $D$ obtained from SVD does one define the trace norm as $\vert \vert A \vert \vert _{Tr} = \sum_{i=1}^r \vert d_i\vert $ ? (where $d_i$ is the $i^{th}$ diagonal element of $D$)

  • But one could also have defined $\vert \vert A \vert \vert _{Tr} = \sum_{i=1}^n \sigma_i(A)$ where $\sigma_i$ is the $i^{th}$ singular value of $A$ defined as $\sigma_i(A) = \sqrt{\lambda_i (A^T A)}$. (where $\lambda_i$ picks out the $i^{th}$ largest eigenvalue of its matrix argument and the $\sqrt{}$ is taking the positive square-root)

Which is the right way to define it?

For which of the two does one get a good bounding by the Frobenius norm?

  • Now using the SVD one would get $A^TA = VD^2V^T$ and it doesn't seem to follow from here that $\sqrt{\lambda_i(VD^2V^T)} = \vert d_i \vert $

  • In the singular value formulation a special case emerges when $A$ is symmetric and then we have $\sigma_i(A) = \vert \lambda_i (A) \vert $. This in the SVD formulation would correspond to $n=m, U =V$ but does it imply anything more in terms of the $d_i$?

  • $\begingroup$ In the SVD factorization, the diagonal entries of $D$ are guaranteed to be nonnegative. $\endgroup$ – littleO Jun 26 '16 at 22:32
  • $\begingroup$ Does the equivalence follow from there? $\endgroup$ – gradstudent Jun 26 '16 at 22:33

First note that in the SVD of $A$, the diagonal entries of $D$ are guaranteed to be nonnegative. So the absolute values are unnecessary in the first definition of the trace norm.

The two definitions of the trace norm are equivalent. Note that $A = UDV^T \implies A^T A = V D^T D V^T \implies A^T A V = V D^T D$, which shows us that the columns of $V$ form a basis of eigenvectors for $A^T A$, and the corresponding eigenvalues are the diagonal entries of $D^T D$. So the singular values of $A$ (the diagonal entries of $D$) are the square roots of eigenvalues of $A^T A$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.