# Confused with this SVD problem: Does it matter which singular vectors you choose?

I am trying to decompose the following matrix using the Singular Value Decomposition (SVD): $$A = \begin{bmatrix} 4 & 4\\ -3 & 3\\ \end{bmatrix} = U\Sigma V^T$$

Here is my work (I know this is far from the most efficient way to do SVD, but please follow along my way):

Finding $$\Sigma$$ and $$V$$:

$$A^T A = \begin{bmatrix}25 & 7\\7 & 25\\\end{bmatrix} \quad\text{with } \lambda_1 = 32, v_1 = \begin{bmatrix}1/\sqrt2\\1/\sqrt2\\\end{bmatrix} \quad\text{and } \lambda_2 = 18, v_2 = \begin{bmatrix}1/\sqrt2\\-1/\sqrt2\\\end{bmatrix}$$

So, $$V = \begin{bmatrix}1/\sqrt2 & 1/\sqrt2\\1/\sqrt2 & -1/\sqrt2\\\end{bmatrix} \quad \text{and} \quad \Sigma = \begin{bmatrix} \sqrt{32} & 0 \\ 0 & \sqrt{18}\\\end{bmatrix}$$

Finding $$U$$:

$$A A^T = \begin{bmatrix}32 & 0\\0 & 18\\\end{bmatrix} \quad\text{with } \lambda_1 = 32, u_1 = \begin{bmatrix}1\\0\\\end{bmatrix} \quad\text{and } \lambda_2 = 18, u_2 = \begin{bmatrix}0\\1\\\end{bmatrix}$$

So, $$U = \begin{bmatrix}32 & 0\\0 & 18\\\end{bmatrix}$$

However, $$U \Sigma V^T = \begin{bmatrix}4 & 4\\3 & -3\\\end{bmatrix} \neq A$$ Did I do something wrong?

Next attempt:

This time, I used $$u_2 = \begin{bmatrix}0\\-1\\\end{bmatrix}$$ instead of $$\begin{bmatrix}0\\1\\\end{bmatrix}$$. So, $$U = \begin{bmatrix}1 & 0\\0 & -1\\\end{bmatrix}$$.

Now, it seems to work: $$U \Sigma V^T = \begin{bmatrix}4 & 4\\-3 & 3\\\end{bmatrix} = A.$$

So my question is: Does it matter which singular vectors you choose for $$U$$ and $$V$$? In other words, if you find a singular vector $$x$$ with unit length, how do you know to choose $$x$$ or $$-x$$? I know that in Eigenvalue decomposition, it didn't matter because you can change the diagonal matrix $$\Lambda$$ accordingly. What about in SVD?

• See this post and this post Commented Aug 8, 2020 at 14:28
• @BenGrossmann Thank you. I think I can now answer my own question Commented Aug 8, 2020 at 14:41
• The SDV decomposition is not unique. Any SDV however has the same values (maybe in different order) for the diagonal matrix $D$. Here is a useful math.stackexchange.com/a/3752311/809021 Commented Aug 8, 2020 at 14:57

Short Answer: The convention of choosing non-negative singular values locks our answer for $$U$$.
Explanation: The square of the singular values ($$\sigma_i$$) of a matrix $$A$$ is equal to the eigenvalues of $$A^T A$$ ($$\lambda_i$$), which are all non-negative (since $$A^T A$$ is positive semidefinite). There are two solutions to $$\sigma_i^2 = \lambda_i$$: the positive root ($$\sigma_i = \sqrt\lambda_i$$), and the negative root ($$\sigma_i = -\sqrt\lambda_i$$). If we were free to choose either one, then we could also choose either $$u_i$$ or $$-u_i$$ as the $$i^{th}$$ left singular vector, since we can adjust the sign of $$\sigma_i$$ in the diagonal matrix $$\Sigma$$ to match the final calculation for $$A = U \Sigma V^T$$. However, since it is a convention to use the positive root in $$\Sigma$$, our choice for $$u_i$$ is locked by $$A$$ and $$v_i$$: $$u_i = \frac{A v_i}{\sigma_i}.$$
In the example above, $$U = \begin{bmatrix} 1 & 0\\ 0 & 1\\ \end{bmatrix}$$ works fine if we choose $$\sigma_2 = -\sqrt{18}$$. Then: $$U \Sigma V^T = \begin{bmatrix} 1 & 0\\ 0 & 1\\ \end{bmatrix} \begin{bmatrix} \sqrt{32} & 0\\ 0 & -\sqrt{18}\\ \end{bmatrix} \begin{bmatrix} 1/\sqrt2 & 1/\sqrt2\\ 1/\sqrt2 & -1/\sqrt2\\ \end{bmatrix} = \begin{bmatrix} 4 & 4\\ -3 & 3\\ \end{bmatrix} = A$$
However, since we choose the positive root for $$\sigma_2$$ by convention, we would have to change the sign of either $$u_2$$ or $$v_2$$ (the latter would flip the sign of the second row of $$V^T$$): $$U \Sigma V^T = \begin{bmatrix} 1 & 0\\ 0 & -1\\ \end{bmatrix} \begin{bmatrix} \sqrt{32} & 0\\ 0 & \sqrt{18}\\ \end{bmatrix} \begin{bmatrix} 1/\sqrt2 & 1/\sqrt2\\ 1/\sqrt2 & -1/\sqrt2\\ \end{bmatrix} = \begin{bmatrix} 4 & 4\\ -3 & 3\\ \end{bmatrix} = A$$
$$\text{or}$$ $$U \Sigma V^T = \begin{bmatrix} 1 & 0\\ 0 & 1\\ \end{bmatrix} \begin{bmatrix} \sqrt{32} & 0\\ 0 & \sqrt{18}\\ \end{bmatrix} \begin{bmatrix} 1/\sqrt2 & 1/\sqrt2\\ -1/\sqrt2 & 1/\sqrt2\\ \end{bmatrix} = \begin{bmatrix} 4 & 4\\ -3 & 3\\ \end{bmatrix} = A$$