I've already read the following related questions on math.SE:

The conclusion they seem to agree on is the following:

For $A \in \mathbb{R}^{m\times n} $, its singular values are real non-negative.

I do not see, however, how can this be the case for $A \in \mathbb{C}^{m\times n}$, even though in most answers, people say it is the same as for real matrices.

Usual argument for singular values of $A \in \mathbb{R}^{m\times n} $ being real non-negative:

The SVD of $A$ is: $$ A = U S V^T $$ We have $$ B = A^T A = V S^T U^T U S V^T = V S^T S V^T $$ where $S^T S$ is diagonal with elements $\sigma_i^2$, where $\sigma_i$ are the singular values of $A$. Now, $\sigma_i^2$ are real-nonegative because they can be seen as the eigenvalues $\lambda_i=\sigma_i^2$ of the symmetric, positive-definite, matrix $B = V \Lambda V^T$ (i.e., $\Lambda = S^T S$).

Using $\lambda_i=\sigma_i^2 \ge 0$ to solve for $\sigma_i$, we find that:

  • $\sigma_i$ is real because if it were complex, then the only way $\sigma_i^2$ would be real is if $\sigma_i$ is real or pure imaginary. However, in the latter case we get $\sigma_i^2$ negative, which contradicts $\sigma_i^2 \ge 0$.
  • $\sigma_i$ is the square-root of $\lambda_i$, which can be positive or negative. By convention, however, we take the positive square-root.

Hence, $$ \sigma_i \text{ are real-nonnegative themselves.} $$

A try for a similar argument for singular values of $A \in \mathbb{C}^{m\times n} $ :

The SVD of $A$ is: $$ A = U S V^H $$ We have $$ B = A^H A = V S^H U^H U S V^H = V S^H S V^H $$ where $S^H S$ is diagonal with elements $|\sigma_i|^2$, where $\sigma_i$ are the singular values of $A$. Now, $|\sigma_i|^2$ are real-nonnegative because of the modulus square, and they can also be seen as the eigenvalues $\lambda_i=|\sigma_i|^2$ of the Hermitian, positive-definite, matrix $B = V \Lambda V^H$ (i.e., $\Lambda = S^H S$).

Using $\lambda_i=|\sigma_i|^2 \ge 0$, how can one solve for $\sigma_i$ and prove it is real-nonnegative?

Particularly, what prevents $\sigma_i$ from being complex?

  • $\begingroup$ Because a complex number times its conjugate is always real and positive. $\endgroup$ – John May 21 '19 at 0:17
  • $\begingroup$ en.m.wikipedia.org/wiki/Gramian_matrix $\endgroup$ – user3417 May 21 '19 at 1:21

The result is that a complex matrix $A$ has a factorization of form $A=USV^H$ where $S$ is nonnegative real diagonal and $U$ and $V$ are "unitary" in the sense that they might be rectangular but $U^HU$ and $V^HV$ are identity matrices. To produce it one takes the positive square roots of the eigenvalues of $AA^H$ and $A^HA$.

No one asserts that all factorizations of $A$ as a product of form $USV^H$ with "unitary" $U$ and $V$ and diagonal $S$ must have all entries of $S$ nonnegative real.

Take, for instance, factorizations of form $A=UDD^{-1}SV^H$ where $D$ is diagonal with complex numbers of unit magnitude on the diagonal. If $U$ is "unitary" so is $UD$, if $S$ is diagonal, so is $D^{-1}S$. So if there is a standard SVD decomposition $A=USV^H$ there are non-standard ones too, that have non-positive diagonal middle factors.

More concretely: suppose $A=USV^H$ is an SVD in the standard sense. Then $A=(-U)(-S)V^H$ is also true, and $-U$ is "unitary" since $U$ is. But $-S$ is not positive on its diagonal. The factorization $A=(-U)(-S)V^H$ holds, but is not a standard SVD factorization.

So the answer boils down to this: the middle factor in an SVD is defined to be nonnegative real diagonal.

  • $\begingroup$ So, nothing prevents $\sigma_i$ from being complex. It's just that we've chosen among the possible choices for $\sigma_i$ one particular choice and called the relative factorization SVD. $\endgroup$ – Likely May 21 '19 at 2:20
  • $\begingroup$ Exactly. The wide range of applications of the SVD constitutes a kind of justification for the definition we've chosen. $\endgroup$ – kimchi lover May 21 '19 at 2:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.