SVD in the language of linear operators?

Question

I noticed that Axler's Linear Algebra Done Right has an explanation of SVD in a matrix free way. The statement of the theorem is the following

Given $T\in L(V)$ there are orthonormal basis $(e_1,\ldots,e_n)$ and $(f_1,\ldots,f_n)$ of $V$ such that $Tv = s_1\langle v,e_1\rangle f_1+\ldots+s_n\langle v,e_n\rangle f_n$, where $s_1,\ldots,s_n$ are the singular values of $T$.

This is easy to prove by just letting $(e_1,\ldots,e_n)$ the an orthonormal basis for $T^*T$, which is guaranteed to have such a basis being self-adjoint. We then apply the polar composition by writing $T=S\sqrt{T^*T}$, where $S$ is an isometry, so it preserves orthonormality of vectors and we can set $f_i = Sf_i$ and seeing that $Tv$ has the required form is then a trivial computation.

My question is the following: In most applications of SVD, where we do some form of dimensionality reduction, we do not have a square matrix, so the assumption $T\in L(V)$ doesn't hold and we would instead need to generalize this to the setting $T\in L(V,W)$, where $V$ and $W$ are vector spaces of possible different dimensions. Is there a nice formulation of the SVD in this setting comparable to the one above?

Answer 1

The cleanest way to extend the matrix-free statement of the SVD to the case of linear maps from $V$ to $W$ is as follows (note the addition of the word "nonzero" and the change from "bases" to "lists"):

Suppose the nonzero singular values of $T \in \mathcal{L}(V, W)$ are $s_1, \dots, s_n$. Then there exist orthonormal lists $e_1, \dots, e_n$ in $V$ and $f_1, \dots, f_n$ in $W$ such that $$ Tv = s_1 \langle v, e_1 \rangle f_1 + \dots + s_n \langle v, e_n \rangle f_n $$ for every $v \in V\!$.