Why the singular values have to appear in descending order across the diagonal matrix?

Question

This question is about Singular Value Decomposition.

Given an arbitrary matrix $A$

$$A \in \mathbb{R}^{m\times n} $$

Its reduced SVD form is:-

$$A = U\Sigma V^T$$

Where as $\Sigma$ is the diagonal matrix containing scaling factors across its diagonal:-

$$ \Sigma = \left( \begin{array}{ccccc} \sigma_1 & \hfill & \hfill & \hfill & \hfill \\ \hfill & \sigma_2 & \hfill & \hfill & \hfill \\ \hfill & \hfill & \ddots &\hfill & \hfill \\ \hfill & \hfill & \hfill & \hfill & \sigma_n \\ \end{array} \right)$$

Now, it is said that the singular values across the diagonal are such that:-

$$\sigma_1 \geq \sigma_2 \geq \sigma_3 \geq \cdots \geq \sigma_n \geq 0 $$

Why do these values have to be necessarily in descending order across the diagonal? I read that it's a convention to write so, but there is more to the order than just convention. If this order is changed, we will get a totally different matrix instead of $A$. So the order is important.

Or is it so that there is always guaranteed to be one unique solution for SVD in which these singular values across $\Sigma$ are guaranteed to be in descending order?

Answer 1

It is just convention. If you change the order, and permute the columns of $U$ and $V$ correspondingly, you do get the same matrix $A$.

Answer 2

Old post, but thought I'd comment anyway. The answer is correct, but there is more to the rankings of singular values (as they decline along the diagonal). The singular values indicate how much INFLUENCE each left vertical (column) * right horizontal row has on the "movement" in the matrix of raw values. If you think in terms of multi-linear regression, it is kind of like the increment/delta in R-squared as you add variables. Lets say you have a 50x50 diagonal (middle) matrix S or "sigma"....you may need to rescale the diagonal by sqrt() or sq() [ie, sq() = ^2]. Then if you sum the diagonal, the "importance" of each left vector/right row is the quotient of it's respective diagonal divided by the diagonal sum. That allows you to say (in the 50x50 example) that the first X (a number, out of 50) column/row pairs account for YY.Y% of the total movement of the matrix data. I'm applied not theory-driven so pls forgive in advance any errors in above (and pls, speak up and correct me if so).