5
$\begingroup$

Let $X$ be the space of all real $n \times n$ matrices, with strictly negative determinant, and pairwise distinct singular values. $X$ is an open subset of the space of all real square matrices. Is there a way to choose smoothly the singular vectors over $X$?

More precisely, I want smooth maps $U:X \to \text{SO}_n,V:X \to \text{O}_n^-$ such that $$ A=U(A)\Sigma(A)V(A)^T,$$ holds for every $A \in X$, where $\Sigma(A) = \text{diag}\left( \sigma_1(A),\dots\sigma_n(A) \right)$, and $\sigma_1(A)$ is the smallest singular value of $A$. (i.e. I specifically want the first diagonal element of $\Sigma(A)$ to be minimal).

Do such smooth maps $U,V$ exist?

Edit: I now see that that if such $U,V$ exist, then the map $A \to \Sigma(A)$ is also smooth. I guess that in general, as long as the singular values are distinct, they can be chosen smoothly (this should probably follow from more general theory of smooth selection of eigenvectors), but I am not sure that one can choose them in a way which keeps the "identity" of the smallest one. Anyway, any reference on that part would be appreciated (although I am really interested in the "harder" question of a possible smooth selection of the singular vectors themselves, not just the singular values).

$\endgroup$
2
+50
$\begingroup$

Let $\mathcal D$ be the space of $n\times n$ diagonal matrices with distinct non-zero positive entries. This has $n!$ connected components corresponding to the ways to order the elements on the diagonal. Fix a connected component $\mathcal D_0\subset \mathcal D.$ I claim that the map

\begin{align*} \mu: SO_n\times \mathcal D_0\times O^-_n\to X\\ (U,\Sigma,V)\mapsto U\Sigma V^T \end{align*}

is a $2^{n-1}$-fold smooth covering map from a connected space (for $n\geq 2$). So it can be inverted locally but has no global section.

$A\mapsto \Sigma(A)$ by itself is a smooth function, because $\Sigma\in\mathcal D_0$ is uniquely determined. It may be worth mentioning that while the $k$'th singular value is not smooth for arbitrary matrices - consider $\mathrm{diag}(1,t)$ as $t\in(0,2)$ - it is Lipschitz continuous in the singular (i.e. operator) norm. See Golub-van Loan, Matrix Computations, Corollary 8.6.2.

Although $\Sigma$ is uniquely determined by $U\Sigma V^T,$ the matrices $U$ and $V$ are not. If $\mu(U',\Sigma,V')=\mu(U,\Sigma,V)$ then $U^{-1}U'$ and $V^{-1}V'$ are equal and of the form $\operatorname{diag}(\pm1,\dots,\pm1)$ with an even number of $-1$'s - see for example the answer at How unique (on non-unique) are U and V in Singular Value Decomposition (SVD)?. This explains the $2^{n-1}.$ In short: let $\hat U=U^{-1}U'$ and $\hat V=V^{-1}V'^T.$ Then $\hat U\Sigma \hat V^T=\Sigma,$ so $\hat U\Sigma^2 \hat U^T=(\hat U\Sigma \hat V^T)(\hat U\Sigma \hat V^T)^T=\Sigma^2,$ which means $\hat U$ commutes with $\Sigma.$ This forces $\hat U$ to be diagonal. Similarly $\hat V$ is diagonal. My intuition for this is that if $A=U\Sigma V^T$ then $U$ has to map standard basis vectors to the corresponding left singular vectors of $A,$ and if we ignore the condition $\det U=1$ for a moment, there are exactly two ways to do that for each vector because only the sign is ambiguous. $V$ also has to map standard basis vectors to right singular vectors of $A,$ but the sign is already determined by the choice of $U.$

To see that it's a local diffeomorphism, use the inverse function theorem. By a symmetry argument it suffices to check the derivative at $U=V=1.$ We need to check that $(u,s,v)\mapsto D\mu(u,s,v)=u\Sigma + s + \Sigma v$ is an injective linear map, where $u,v$ are skew-symmetric and $s$ is diagonal. The diagonal entries are just those of $s.$ The $i,j$ entry is $u_{ij} \sigma(j)+\sigma(i) v_{ij},$ and the $j,i$ entry is $u_{ji} \sigma(i)+\sigma(j) v_{ji}=-u_{ij} \sigma(i)-\sigma(j) v_{ij}.$ Since $\begin{pmatrix}\sigma(j)&\sigma(i)\\-\sigma(i)&-\sigma(j)\end{pmatrix}$ is invertible, we can recover $u_{ij}$ and $v_{ij}$ from these.

Finally, $\mu$ is a covering map because each fiber has the same (finite) cardinality. Indeed for any $x\in X$ by the local homeomorphism and Hausdorff properties, there are disjoint open sets $U_1,\dots,U_{2^{n-1}}$ each mapped diffeomorphically by $\mu$ to a (possibly different) open neighborhood of $x.$ This implies that $\bigcap_i\mu(U_i)$ is an evenly covered neighborhood of $x.$


The way I originally thought of this problem is to consider the map $f:SO_2\to \mathbb R^{2\times 2}$ that sends $U$ to $U\cdot \mathrm{diag}(2,1)\cdot U^T.$ This is a slightly different situation, but perhaps a bit easier to visualize. $f(U)$ is a positive definite matrix, and $x^Tf(U)x=1$ describes a certain ellipse with axes of length 1 and $1/\sqrt 2.$ The question is whether we can smoothly choose the rotation matrix $U,$ given the ellipse. This is impossible because continuously rotating by 180 degrees gives the same ellipse, but forces $U$ to end up with an extra 180 degree rotation. Actually $f:SO_2\to f(SO_2)$ is just the map $S^1\to S^1$ of degree $2.$

This argument relies on the fact that the fibers are discrete. Otherwise you'd get a more general fibration.

$\endgroup$
  • $\begingroup$ Thank you! This is a really impressive, beautiful and illuminating answer. This is not the first time you are giving me such insightful answers, and I am really grateful. I have a few comments, to make sure that I understand: (1) I only assumed that the first diagonal element $\sigma_1$ is the smallest one, not that all the singular values are ordered in a strictly increasing manner. I guess the question can be reduced to this case, though: Since the $\sigma_i(A)$ remain distinct, while they change continuously with $A$, they cannot "cross" (since the domain is connected);... $\endgroup$ – Asaf Shachar Mar 27 at 14:37
  • $\begingroup$ i.e. the relative order must be fixed. Thus W.L.O.G we can assume it is strictly increasing. (e.g. by padding with permutation matrices). Was that your thinking? (2) Was your original argument for showing that the orthogonal matrices fixes the standard basis vectors based on comparison of norms (after we act on vectors)? (here we use the fact the singular values are distinct). (3) I think that the properness is actually not hard: The orthogonal components always converge after passing to a subsequence, and the middle part follows. $\endgroup$ – Asaf Shachar Mar 27 at 14:51
  • $\begingroup$ (4) The idea that this is a covering map is really cool! Was there any particular source of inspiration (or: how did you think about it?) (5) Maybe you will be interested to know the source of this question: I tried to find out wether a certain minimizer behaves smoothly. (here:mathoverflow.net/questions/326374/…). In particular, the local invertibility you established implies this. (6) Finally, do you have any idea whether or not the singular values (only them) can be chosen globally in a continuous manner? $\endgroup$ – Asaf Shachar Mar 27 at 14:51
  • $\begingroup$ I meant to reply to this properly but had to run - I'll try to respond by next week. $\endgroup$ – Dap Mar 29 at 11:24
  • $\begingroup$ Thank you. After some more thought, I think I can answer two of my points above: Regarding the covering map, I guess that after you see that the fibers are of constant size, it is a rather natural guess (I guess that most "natural" local diffeomorphisms with fibers of constant size are coverings). I also see now that the the singular values can be chosen globally in a continuous manner - you can do this locally via your argument, but all these local pieces must glue together to a consistent choice-which is forced upon us by the order. $\endgroup$ – Asaf Shachar Mar 31 at 12:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.