Singular Value Decomposition in Axler's book

Question

In Axler's "Linear Algebra Done Right", he gave the singular-value decomposition as:

$Tv = s_1\langle v,e_1\rangle f_1 + \cdots + s_n\langle v,e_n\rangle f_n$, where T is an operator; $s_1,\ldots,s_n$ are the singular values of $T$; $(e_1,\ldots,e_n)$ and $(f_1,\ldots,f_n)$ are orthonormal bases of $V$. Singular values of $T$ are defined as eigenvalues of $\sqrt{T^*T}$. $\langle ,\rangle $ is inner product.

I am having trouble connecting this to the more generally used matrix representation $A=UΣV^T$. Matrices in Axler's book are considered as linear maps, "with respect to some bases". How do I understand Axler's exposition using matrices?

Thank you.

Update: To clarify, I was thinking from the perspective where an operator is a matrix with respect to some basis (in his book...). So an operator can be written as $M(T,(b1,...bn),(b1,...bn))$. If I apply change of basis, I get $M(I,(f1,...fn),(b1,...bn))*M(T,(e1,...en),(f1,...fn))*M(I,b1,...bn),(e1,...en))$ where (e1,...en) is an orthonormal basis; (f1,...fn) is another orthonormal basis given by $fi=Sei$ given by isometry $S$ from polar decomposition.

$M(T,(e1,...en),(f1,...fn))$ gives me the diagonol matrix $Σ$. But how about the rest? Thanks.

Update 2: Thanks for the answers below and I understand those. But my question is a bit specific to Axler's definition. In Axler's book, he thinks of matrix a bit differently: m*n matrix is defined as $M(T,(v_1...v_n),(w_1..w_m))$ where $T$ is a linear map from space $V$ to W, and $(v_1...v_n)$ and $(w_1...w_m)$ are bases for 2 spaces. For the $k$th column in the matrix, each entry $a_{ik}$ is defined as $Tv_k=a_{1k}w_1+...a_{mk}w_m$, so the matrix represents the linear map between the bases. So it's like "you don't talk about a matrix without talking about $T$ and the bases". Whether this is a good way to think about matrix is probably also a topic for debate...

Anyway, now with that, I am trying to decompose: $M(T,(b_1,...b_n),(b_1,...b_n))$. I already know that the diagonol $Σ$ in SVD can be thought of as: A. $M(T,(e_1,...e_n),(f_1,...f_n))$, (where $f_i=Se_i$ given by isometry $S$ from polar decomposition $T=S\sqrt{T^*T}$), OR, B. $M(\sqrt{T^*T},(e_1,...e_n),(e_1,...e_n))$. So I would also like to know: 1. the operator 2. the bases, for the other 2 unitary matrices, no matter which one you choose for $Σ$.

Thanks again.

Answer 1

You can expand in coordinates. The image vectors need to be column vectors with respect to the chosen basis. So, you want the first index to loop in order to think of what you have as part of a matrix. In other words, $f_{1,j}$ is the first coordinate of $f_{j}$, $f_{2,j}$ is the second coordinate of $f_{j}$ with respect to the chosen basis, etc.. This leads to the coorespondence $$ f_{j} = \left[\begin{array}{c}f_{1,j} \\ f_{2,j} \\ f_{3,j} \\ \vdots \\ f_{N,j}\end{array}\right] $$ Now you can write $$ \alpha_{1}f_{1}+\cdots+\alpha_{n}f_{N} = \\ = \alpha_{1} \left[\begin{array}{c}f_{1,1} \\ f_{2,1} \\ f_{3,1} \\ \vdots \\ f_{N,1}\end{array}\right] + \alpha_{2} \left[\begin{array}{c}f_{1,2} \\ f_{2,2} \\ f_{3,2} \\ \vdots \\ f_{N,2}\end{array}\right]+\cdots+\alpha_{N} \left[\begin{array}{c}f_{1,N} \\ f_{2,N} \\ f_{3,N} \\ \vdots \\ f_{N,N}\end{array}\right] \\ = \left[\begin{array}{cccc} f_{1,1} & f_{1,2} & \cdots & f_{1,N} \\ f_{2,1} & f_{2,2} & \cdots & f_{2,N} \\ \vdots & \vdots & \ddots & \vdots \\ f_{N,1} & f_{N,2} & \cdots & f_{N,N} \end{array}\right]\left[\begin{array}{c}\alpha_{1} \\ \alpha_{2} \\ \vdots \\ \alpha_{N}\end{array}\right] $$ Then you can write $$ \left[\begin{array}{c}\alpha_{1} \\ \alpha_{2} \\ \vdots \\ \alpha_{N}\end{array}\right] = \left[\begin{array}{c}s_{1}\langle v,e_{1}\rangle \\ s_{2}\langle v,e_{2}\rangle \\ \vdots \\ s_{N}\langle v,e_{N}\rangle\end{array}\right] \\ = \left[\begin{array}{cccc} s_1 & 0 & 0 & \cdots & 0\\ 0 & s_2 & 0 & \cdots & 0 \\ 0 & 0 & s_3 & \cdots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \cdots & s_N \end{array}\right] \left[\begin{array}{c}\langle v,e_1\rangle \\ \langle v,e_2\rangle \\ \langle v, e_3 \rangle \\ \vdots \\ \langle v,e_N \rangle\end{array}\right] $$ For the last part, you have a column vector representation of $v$ with respect to your standard basis with column coordinates $v_1,v_2,\cdots,v_N$, and you have the vectors $e_1,e_2,\cdots,e_N$ as row vectors with $e_1=[e_{1,1}\; e_{1,2}\; e_{1,3}\; \cdots \;e_{1,N}]$ in order to view the dot product a row vector on the left times a column vector on the right. That way you can write $$ \left[\begin{array}{c}\langle v,e_1\rangle \\ \langle v,e_2\rangle \\ \langle v, e_3 \rangle \\ \vdots \\ \langle v,e_N \rangle\end{array}\right] = \left[\begin{array}{cccc} e_{1,1} & e_{1,2} & e_{1,3} & \cdots & e_{1,N} \\ e_{2,1} & e_{2,2} & e_{2,3} & \cdots & e_{2,N} \\ e_{3,1} & e_{3,2} & e_{3,3} & \cdots & e_{3,N} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ e_{N,1} & e_{N,2} & e_{N,3} & \cdots & e_{N,N} \end{array}\right] \left[\begin{array}{c}v_1 \\ v_2 \\ v_3 \\ \vdots \\ v_N\end{array}\right] $$ Now you can see that what you have is $U\Sigma V^{T}$ where $U$ has columns vectors $f_{j}$ and $V$ has column vectors $e_{j}$.

Answer 2

Let \begin{equation*} V = \begin{bmatrix} e_1 & \cdots & e_n \end{bmatrix}, U = \begin{bmatrix} f_1 & \cdots & f_n \end{bmatrix}, \Sigma = \begin{bmatrix} s_1 & & \\ & \ddots & \\ & & s_n \end{bmatrix}. \end{equation*}

First note that \begin{equation*} V^T v = \begin{bmatrix} \langle v, e_1 \rangle \\ \vdots \\\langle v, e_n \rangle \end{bmatrix}. \end{equation*}

So, \begin{equation} \Sigma V^T v = \begin{bmatrix} s_1 \langle v, e_1 \rangle \\ \vdots \\ s_n \langle v, e_n \rangle \end{bmatrix} . \end{equation} Finally, \begin{equation} U \Sigma V^T v = s_1 \langle v, e_1 \rangle f_1 + \cdots + s_n \langle v, e_n \rangle f_n. \end{equation} (The last step is easy if you remember that when you multiply a matrix by a vector, the output is a linear combination of the columns of the matrix.)

Answer 3

Remember in Axler's book, any linear operator can be represented by a diagonal matrix with singular values on the diagonal, with respect to two orthonormal basis from $e_i$ to $f_i$. Before applying this matrix, we first need to make sure the vector is measured with $e_i$ as basis. After applying the matrix, we may want to convert the vector from $f_i$ back to its original basis. Thus the matrix before and after the diagonal matrix are the change of basis matrix.