rotationally invariant matrix function Consider a function $f:\mathbb{R}^{N \times M} \to \mathbb{R}^{N \times M}$, that takes a matrix $\mathbf{A} \in \mathbb{R}^{N \times M}$ as input and the output is a matrix of the same size.
Suppose that the function $f$, has the following property that for any orthogonal matrices $\mathbf{O} \in \mathbb{R}^{N \times N}$, $\mathbf{V} \in \mathbb{R}^{M \times M}$, we have that:
$$
\mathbf{O} f(\mathbf{A})\mathbf{V}^T = f(\mathbf{O} \mathbf{A} \mathbf{V}^T)
$$
Is this condition sufficient to conclude that $f(\mathbf{A})$ has the same singular vectors (both left and right) as the matrix $\mathbf{A}$? Or there is a counterexample?
 A: No. Every singular vector of $A$ is a singular vector of $f(A)$, but the converse is not true. For a simple counterexample, consider $f=0$ and any $A$ that has at least two different singular values.
Let $\Sigma=\sigma_1I_{r_1}\oplus\sigma_2I_{r_2}\oplus\cdots\oplus\sigma_kI_{r_k}\oplus0$ be a singular value matrix, where $\sigma_1>\sigma_2>\cdots>\sigma_k>0$. By assumption, we have
$$
\pmatrix{Q_1\\ &Q_2\\ &&\ddots\\ &&&Q_k\\ &&&&U'}
f(\Sigma)
\pmatrix{Q_1^T\\ &Q_2^T\\ &&\ddots\\ &&&Q_k^T\\ &&&&V'}
=f(\Sigma)
$$
for every $Q_i\in O(r_i,\mathbb R)$ and every pair orthogonal matrices $U',V'$ of appropriate sizes. It follows that $f(\Sigma)$ must be a diagonal matrix in the form of
$$
\lambda_1I_{r_1}\oplus\lambda_2I_{r_2}\oplus\cdots\oplus\lambda_kI_{r_k}\oplus0.
$$
Therefore, given any two orthogonal matrices $U\in O(N,\mathbb R)$ and $V\in O(M,\mathbb R)$, we have
$$
f(U\Sigma V^T)
=Uf(\Sigma)V^T
=U(\lambda_1I_{r_1}\oplus\lambda_2I_{r_2}\oplus\cdots\oplus\lambda_kI_{r_k}\oplus0)V^T.
$$
While this is not immediately in the form of a SVD (because some $\lambda_i$s may be negative), the columns of $V$ are still right singular vectors of $f(U\Sigma V^T)$ and the columns of $U$ are still left singular vectors of $f(U\Sigma V^T)$. However, as it can happen that $\lambda_i=\lambda_j$ or $\lambda_i=0$ for some $i\ne j$, in which case $f(U\Sigma V^T)$ has fewer but larger singular spaces than $U\Sigma V^T$ does.
