# Why the singular values in SVD are always hierarchical/descending?

Please, I'm trying to understand why singular values (SV) are always hierarchical/descending. At the beginning of my studies, I thought that the hierarchy of sigmas ($$\sigma_1 \geq \sigma_2 \geq ... \geq \sigma_r$$) was a convention/premise of SVD, but it seems to be a consequence of the factorization itself.

In order words:

Why it is not possible to get $$\sigma_1 < \sigma_2$$ in SVD? Because the hierarchy of SV is the whole key idea of SVD and PCA.

Brainstorming (my hypothesis):

I've been thinking about it for a while. The answer might has to do with the first optimal rotation angle from the $$V^T$$ matrix ( related to $$v_1^T$$ ). The stretching transformation ( $$\Sigma$$ ) is a "static" transformation, sequentially after the first main rotation ( $$V^T$$ ). The order might come due the way $$V^T$$ matrix has to be. What do you think? Does it sound reasonable to you?

Trying to get a help from chatGPT. I asked:

"Is the descending order of the singular values in the singular values decomposition method just a convention? "

ChatGTP answer: "No, the descending order of the singular values in the singular value decomposition (SVD) method is not just a convention. It arises naturally from the mathematical properties of the SVD. The singular values are the square roots of the eigenvalues of the covariance matrix of the original data, and eigenvalues are always ordered in descending magnitude. Therefore, it makes sense to order the singular values in descending order to reflect the magnitude of the corresponding singular vectors in the approximation of the original matrix."

• I think you got a typo in the hierarchy in your question, it should be descending. Commented Feb 5, 2023 at 17:54
• Thank you, it might be even more appropriate indeed! I used hierarchical because I used the same terminology as a prof. from Univ. Washington (Steve L. Brunton). But I'll gladly edit the title and incorporate the more precise term there. Thanks! Commented Feb 5, 2023 at 17:58
• I actually meant in your sigmas, it should be $\sigma_1>\sigma_2$ in one part of your question right now it is $\sigma_1<\sigma_2$ in both places. Commented Feb 5, 2023 at 18:04
• There is no particular order among singular values. It's just a convention to sort the matrices that way. You can permute rows columns and diagonals and keep the same matrix if you want to. Commented Feb 5, 2023 at 19:28
• @Caio The hierarchy of sigmas is absolutely just a convention. There are several reasons that this convention is convenient, but there is nothing stopping you from producing something like the SVD except that the sigmas are in a different order. ChatGPT is great at writing text that looks correct, but doesn't always write things that are true. Commented Feb 5, 2023 at 19:53

There is no such magical order. Assume we have a singular value decomposition of a matrix $$M=U\Sigma V^T$$
You can devise any permutation $$P$$ and use $$P^{-1} = P^{T}$$ So that $$PP^T = P^TP = I$$
$$M={\underset{U_2}{(U P)}}\underset{\Sigma_2}{(P^T \Sigma P)}\underset{V_2^T}{(P^T V^T)}$$ You have three new matrices and the new $$\Sigma_2 = P^T\Sigma P$$
A curious student can prove that $$U_2,\Sigma_2,V_2$$ will keep the properties demanded of them to qualify as factors for the SVD.