7
$\begingroup$

I noticed that Axler's Linear Algebra Done Right has an explanation of SVD in a matrix free way. The statement of the theorem is the following

Given $T\in L(V)$ there are orthonormal basis $(e_1,\ldots,e_n)$ and $(f_1,\ldots,f_n)$ of $V$ such that $Tv = s_1\langle v,e_1\rangle f_1+\ldots+s_n\langle v,e_n\rangle f_n$, where $s_1,\ldots,s_n$ are the singular values of $T$.

This is easy to prove by just letting $(e_1,\ldots,e_n)$ the an orthonormal basis for $T^*T$, which is guaranteed to have such a basis being self-adjoint. We then apply the polar composition by writing $T=S\sqrt{T^*T}$, where $S$ is an isometry, so it preserves orthonormality of vectors and we can set $f_i = Sf_i$ and seeing that $Tv$ has the required form is then a trivial computation.

My question is the following: In most applications of SVD, where we do some form of dimensionality reduction, we do not have a square matrix, so the assumption $T\in L(V)$ doesn't hold and we would instead need to generalize this to the setting $T\in L(V,W)$, where $V$ and $W$ are vector spaces of possible different dimensions. Is there a nice formulation of the SVD in this setting comparable to the one above?

$\endgroup$
4
  • 1
    $\begingroup$ Maybe the formula $\sum \sigma_ju_jv_j^*$ in page 6 of math.vt.edu/people/embree/cmda3606/chapter8.pdf is what you want ? $\endgroup$
    – paf
    Jun 2, 2017 at 16:32
  • 1
    $\begingroup$ Yes, it's the same except $f_i$ is now an orthonormal basis of $W$. $\endgroup$ Jun 2, 2017 at 16:52
  • $\begingroup$ I'm not sure I follow. We can clearly choose an orthonormal basis of eigenvectors for $T^*T$. However, we're summing over the length of that basis which will be different, if $W$ is lower or higher-dimensional? $\endgroup$
    – eof
    Jun 2, 2017 at 17:08
  • $\begingroup$ It depends on how you index the singular values. If you index the singular values so that some of them are zero, then the singular vectors corresponding to zero singular values are highly non-unique; you basically pick them arbitrarily so that you continue to have an orthonormal basis. If $\dim V = n$ and $\dim W = m$ there are at most $\text{min}(n, m)$ nonzero singular values (in fact there are exactly $\text{rank}(T)$ nonzero singular values), so one convention is to only index up this far. $\endgroup$ Jun 3, 2017 at 4:46

1 Answer 1

4
$\begingroup$

The cleanest way to extend the matrix-free statement of the SVD to the case of linear maps from $V$ to $W$ is as follows (note the addition of the word "nonzero" and the change from "bases" to "lists"):

Suppose the nonzero singular values of $T \in \mathcal{L}(V, W)$ are $s_1, \dots, s_n$. Then there exist orthonormal lists $e_1, \dots, e_n$ in $V$ and $f_1, \dots, f_n$ in $W$ such that $$ Tv = s_1 \langle v, e_1 \rangle f_1 + \dots + s_n \langle v, e_n \rangle f_n $$ for every $v \in V\!$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .