# SVD Singular Value Issue

I am having an issue with a SVD problem and would like advice. The matrices given are

$$A= \left[ \begin{matrix} 1 & 2\\ 3 & 6 \\ \end{matrix}\right]\ \ \ A^TA= \left[ \begin{matrix} 10 & 20\\ 20 & 40 \\ \end{matrix}\right]\ \ \ AA^T= \left[ \begin{matrix} 5 & 15\\ 15 & 45 \\ \end{matrix}\right]$$

I initially determined my $$λ$$ values to be $$λ_1=50$$ and $$λ_2=0$$ by computing the following:

$$det⁡(A^TA-λI)=0\\ \begin{vmatrix} 10-λ & 20\\ 20 & 40-λ \\ \end{vmatrix}\\ (10-λ)(40-λ)-20^2=0\\ 400-50λ+λ^2-400=0\\ λ^2-50λ=0\\ λ(λ-50)=0$$

This in turn means that my singular values will be $$\sigma_1=\sqrt {50}$$ and $$\sigma_2=0$$, and this is where I run into my issue. When I am trying to solve for the orthonormal set with $$u_1$$ and $$u_2$$, I was told to use the equation $$u_n=\frac{Av_n}{\sigma_n}$$. I cannot compute $$u_2$$ because that would make the denominator $$0$$, and the fraction would become undefined.

I discussed with my professor and they said I should try to calculate $$u_2$$ using the eigenvectors from $$AA^T$$, but $$AA^T$$ and $$A^TA$$ have the same eigenvalues. I thought maybe they meant to compute the $$u$$ for the $$\sigma=\sqrt {50}$$, but this is not the same answer as the one the book gives. The answers for this and Wolfram Alpha show that the SVD should be

$$\left[ \begin{matrix} 1 & 2\\ 3 & 6 \\ \end{matrix}\right]= \frac{1}{\sqrt{10}} \left[ \begin{matrix} 1 & -3\\ 3 & 1 \\ \end{matrix}\right] \left[ \begin{matrix} \sqrt{50} & 0\\ 0 & 0 \\ \end{matrix}\right] \frac{1}{\sqrt{5}} \left[ \begin{matrix} 1 & -2\\ 2 & -1 \\ \end{matrix}\right]$$

I have gotten that $$u_1=\frac{1}{\sqrt{10}} \left[ \begin{matrix} 1 \\ 3 \\ \end{matrix}\right]$$, but I do not know how to get $$u_2=\frac{1}{\sqrt{10}} \left[ \begin{matrix} -3 \\ 1 \\ \end{matrix}\right]$$. Any help would be appreciated.

## 2 Answers

Your professor was right.

Have you tried the definition of an eigenvector?

$$(AA^T)v=\lambda v$$

$$\left[ \begin{matrix} 5 & 15\\ 15 & 45 \\ \end{matrix}\right] \left[ \begin{matrix} v_1\\ v_2\\ \end{matrix}\right] =\left[ \begin{matrix} 0\\ 0\\ \end{matrix}\right]$$

\begin{align} & 5v_1+15v_2=0\\ & 15v_1+45v_2=3(5v_1+15v_2)=0 \end{align}

So you have one degree of freedom to choose $$v_2 = 1/\sqrt(10)$$ and obtain $$v_1 = -15/5 \, v_2 = -3/\sqrt(10)$$.

Note that another way to write the singular value decomposition is to write it as the sum of rank one matrices: $$A = U D V^T = \sum_{k = 1}^n \sigma_i \vec{u}_i \vec{v}_i^T$$ where $$\vec{u}_i$$ is the $$i$$th column of $$U$$ and $$\vec{v}_i$$ is the $$i$$th column of $$V$$. What does this tell us? It tells us that whenever $$\sigma_i = 0$$, the resulting left and right singular vectors don't really matter. So you have two choices:

1. If $$A$$ is $$m \times n$$ (here $$m = n = 2$$) You can complete $$\vec{u}_1$$ to an orthonormal basis $$\{\vec{u}_1, \vec{u}_2, \cdots, \vec{u}_m\}$$ of $$\mathbb{R}^m$$ and complete $$\vec{v}_1$$ to an orthonormal basis $$\{\vec{v}_1, \vec{v}_2, \cdots, \vec{v}_m\}$$ of $$\mathbb{R}^m$$ of $$\mathbb{R}^n$$ (this can be done with Gram-Schmidt). As a result, $$D$$ will be an $$m \times n$$ diagonal matrix. That's what the solutions have done. Then we may write $$A = \begin{bmatrix} \vec{u}_1 & \cdots & \vec{u}_m \end{bmatrix} \cdot \text{diag}_{m \times n}(\sigma_1, \sigma_2, \cdots, \sigma_{\min(m,n)}) \cdot \begin{bmatrix} \vec{v}_1 & \cdots & \vec{v}_n \end{bmatrix}^T$$

2. Calculate the reduced singular value decomposition. This involves only worrying about the nonzero singular values, so we won't need to complete any set of singular vectors to a basis. Here $$D$$ will become an $$r \times r$$ diagonal matrix, where $$r$$ is the number of nonzero singular values ($$r$$ is also the rank of $$A$$). $$U$$ will be $$n \times r$$, and $$V$$ will be $$m \times r$$. In this case, we know $$r = 1$$ and you've calculated the corresponding left and right singular vectors of $$\sigma_1 = \sqrt {50}$$. So you can write $$A = \frac{1}{\sqrt{10}}\begin{bmatrix} 1 \\ 3\end{bmatrix} \begin{bmatrix} \sqrt{50} \end{bmatrix} \left(\frac{1}{\sqrt 5} \begin{bmatrix} 1 & -2\end{bmatrix}\right)$$ Here $$U$$ and $$V$$ are partially orthogonal, in the sense that $$U^T U = I_r = V^T V$$ but $$UU^T \neq I_n$$ and $$V V^T \neq I_m$$.