Let $\mathcal D$ be the space of $n\times n$ diagonal matrices with distinct non-zero positive entries. This has $n!$ connected components corresponding to the ways to order the elements on the diagonal. Fix a connected component $\mathcal D_0\subset \mathcal D.$ I claim that the map
\begin{align*}
\mu: SO_n\times \mathcal D_0\times O^-_n\to X\\
(U,\Sigma,V)\mapsto U\Sigma V^T
\end{align*}
is a $2^{n-1}$-fold smooth covering map from a connected space (for $n\geq 2$). So it can be inverted locally but has no global section.
$A\mapsto \Sigma(A)$ by itself is a smooth function, because $\Sigma\in\mathcal D_0$ is uniquely determined. It may be worth mentioning that while the $k$'th singular value is not smooth for arbitrary matrices - consider $\mathrm{diag}(1,t)$ as $t\in(0,2)$ - it is Lipschitz continuous in the singular (i.e. operator) norm. See Golub-van Loan, Matrix Computations, Corollary 8.6.2.
Although $\Sigma$ is uniquely determined by $U\Sigma V^T,$ the matrices $U$ and $V$ are not. If $\mu(U',\Sigma,V')=\mu(U,\Sigma,V)$ then $U^{-1}U'$ and $V^{-1}V'$ are equal and of the form $\operatorname{diag}(\pm1,\dots,\pm1)$ with an even number of $-1$'s - see for example the answer at How unique (on non-unique) are U and V in Singular Value Decomposition (SVD)?. This explains the $2^{n-1}.$ In short: let $\hat U=U^{-1}U'$ and $\hat V=V^{-1}V'^T.$ Then $\hat U\Sigma \hat V^T=\Sigma,$ so $\hat U\Sigma^2 \hat U^T=(\hat U\Sigma \hat V^T)(\hat U\Sigma \hat V^T)^T=\Sigma^2,$ which means $\hat U$ commutes with $\Sigma.$ This forces $\hat U$ to be diagonal. Similarly $\hat V$ is diagonal. My intuition for this is that if $A=U\Sigma V^T$ then $U$ has to map standard basis vectors to the corresponding left singular vectors of $A,$ and if we ignore the condition $\det U=1$ for a moment, there are exactly two ways to do that for each vector because only the sign is ambiguous. $V$ also has to map standard basis vectors to right singular vectors of $A,$ but the sign is already determined by the choice of $U.$
To see that it's a local diffeomorphism, use the inverse function theorem. By a symmetry argument it suffices to check the derivative at $U=V=1.$ We need to check that $(u,s,v)\mapsto D\mu(u,s,v)=u\Sigma + s + \Sigma v$ is an injective linear map, where $u,v$ are skew-symmetric and $s$ is diagonal. The diagonal entries are just those of $s.$ The $i,j$ entry is $u_{ij} \sigma(j)+\sigma(i) v_{ij},$ and the $j,i$ entry is $u_{ji} \sigma(i)+\sigma(j) v_{ji}=-u_{ij} \sigma(i)-\sigma(j) v_{ij}.$ Since $\begin{pmatrix}\sigma(j)&\sigma(i)\\-\sigma(i)&-\sigma(j)\end{pmatrix}$ is invertible, we can recover $u_{ij}$ and $v_{ij}$ from these.
Finally, $\mu$ is a covering map because each fiber has the same (finite) cardinality. Indeed for any $x\in X$ by the local homeomorphism and Hausdorff properties, there are disjoint open sets $U_1,\dots,U_{2^{n-1}}$ each mapped diffeomorphically by $\mu$ to a (possibly different) open neighborhood of $x.$ This implies that $\bigcap_i\mu(U_i)$ is an evenly covered neighborhood of $x.$
The way I originally thought of this problem is to consider the map $f:SO_2\to \mathbb R^{2\times 2}$ that sends $U$ to $U\cdot \mathrm{diag}(2,1)\cdot U^T.$ This is a slightly different situation, but perhaps a bit easier to visualize. $f(U)$ is a positive definite matrix, and $x^Tf(U)x=1$ describes a certain ellipse with axes of length 1 and $1/\sqrt 2.$ The question is whether we can smoothly choose the rotation matrix $U,$ given the ellipse. This is impossible because continuously rotating by 180 degrees gives the same ellipse, but forces $U$ to end up with an extra 180 degree rotation. Actually $f:SO_2\to f(SO_2)$ is just the map $S^1\to S^1$ of degree $2.$
This argument relies on the fact that the fibers are discrete. Otherwise you'd get a more general fibration.
