4
$\begingroup$

I am trying to decompose the following matrix using the Singular Value Decomposition (SVD): $$A = \begin{bmatrix} 4 & 4\\ -3 & 3\\ \end{bmatrix} = U\Sigma V^T$$

Here is my work (I know this is far from the most efficient way to do SVD, but please follow along my way):

Finding $\Sigma$ and $V$:

$$A^T A = \begin{bmatrix}25 & 7\\7 & 25\\\end{bmatrix} \quad\text{with } \lambda_1 = 32, v_1 = \begin{bmatrix}1/\sqrt2\\1/\sqrt2\\\end{bmatrix} \quad\text{and } \lambda_2 = 18, v_2 = \begin{bmatrix}1/\sqrt2\\-1/\sqrt2\\\end{bmatrix}$$

So, $$V = \begin{bmatrix}1/\sqrt2 & 1/\sqrt2\\1/\sqrt2 & -1/\sqrt2\\\end{bmatrix} \quad \text{and} \quad \Sigma = \begin{bmatrix} \sqrt{32} & 0 \\ 0 & \sqrt{18}\\\end{bmatrix} $$

Finding $U$:

$$A A^T = \begin{bmatrix}32 & 0\\0 & 18\\\end{bmatrix} \quad\text{with } \lambda_1 = 32, u_1 = \begin{bmatrix}1\\0\\\end{bmatrix} \quad\text{and } \lambda_2 = 18, u_2 = \begin{bmatrix}0\\1\\\end{bmatrix}$$

So, $$U = \begin{bmatrix}32 & 0\\0 & 18\\\end{bmatrix} $$

However, $$U \Sigma V^T = \begin{bmatrix}4 & 4\\3 & -3\\\end{bmatrix} \neq A$$ Did I do something wrong?


Next attempt:

This time, I used $u_2 = \begin{bmatrix}0\\-1\\\end{bmatrix}$ instead of $\begin{bmatrix}0\\1\\\end{bmatrix}$. So, $U = \begin{bmatrix}1 & 0\\0 & -1\\\end{bmatrix}$.

Now, it seems to work: $$U \Sigma V^T = \begin{bmatrix}4 & 4\\-3 & 3\\\end{bmatrix} = A.$$

So my question is: Does it matter which singular vectors you choose for $U$ and $V$? In other words, if you find a singular vector $x$ with unit length, how do you know to choose $x$ or $-x$? I know that in Eigenvalue decomposition, it didn't matter because you can change the diagonal matrix $\Lambda$ accordingly. What about in SVD?

$\endgroup$
3
  • $\begingroup$ See this post and this post $\endgroup$ Commented Aug 8, 2020 at 14:28
  • $\begingroup$ @BenGrossmann Thank you. I think I can now answer my own question $\endgroup$
    – 5Pack
    Commented Aug 8, 2020 at 14:41
  • $\begingroup$ The SDV decomposition is not unique. Any SDV however has the same values (maybe in different order) for the diagonal matrix $D$. Here is a useful math.stackexchange.com/a/3752311/809021 $\endgroup$
    – Mittens
    Commented Aug 8, 2020 at 14:57

1 Answer 1

1
$\begingroup$

Short Answer: The convention of choosing non-negative singular values locks our answer for $U$.


Explanation: The square of the singular values ($\sigma_i$) of a matrix $A$ is equal to the eigenvalues of $A^T A$ ($\lambda_i$), which are all non-negative (since $A^T A$ is positive semidefinite). There are two solutions to $\sigma_i^2 = \lambda_i$: the positive root ($\sigma_i = \sqrt\lambda_i$), and the negative root ($\sigma_i = -\sqrt\lambda_i$). If we were free to choose either one, then we could also choose either $u_i$ or $-u_i$ as the $i^{th}$ left singular vector, since we can adjust the sign of $\sigma_i$ in the diagonal matrix $\Sigma$ to match the final calculation for $A = U \Sigma V^T$. However, since it is a convention to use the positive root in $\Sigma$, our choice for $u_i$ is locked by $A$ and $v_i$: $$u_i = \frac{A v_i}{\sigma_i}.$$

In the example above, $U = \begin{bmatrix} 1 & 0\\ 0 & 1\\ \end{bmatrix}$ works fine if we choose $\sigma_2 = -\sqrt{18}$. Then: $$U \Sigma V^T = \begin{bmatrix} 1 & 0\\ 0 & 1\\ \end{bmatrix} \begin{bmatrix} \sqrt{32} & 0\\ 0 & -\sqrt{18}\\ \end{bmatrix} \begin{bmatrix} 1/\sqrt2 & 1/\sqrt2\\ 1/\sqrt2 & -1/\sqrt2\\ \end{bmatrix} = \begin{bmatrix} 4 & 4\\ -3 & 3\\ \end{bmatrix} = A$$

However, since we choose the positive root for $\sigma_2$ by convention, we would have to change the sign of either $u_2$ or $v_2$ (the latter would flip the sign of the second row of $V^T$): $$U \Sigma V^T = \begin{bmatrix} 1 & 0\\ 0 & -1\\ \end{bmatrix} \begin{bmatrix} \sqrt{32} & 0\\ 0 & \sqrt{18}\\ \end{bmatrix} \begin{bmatrix} 1/\sqrt2 & 1/\sqrt2\\ 1/\sqrt2 & -1/\sqrt2\\ \end{bmatrix} = \begin{bmatrix} 4 & 4\\ -3 & 3\\ \end{bmatrix} = A$$

$$\text{or}$$ $$U \Sigma V^T = \begin{bmatrix} 1 & 0\\ 0 & 1\\ \end{bmatrix} \begin{bmatrix} \sqrt{32} & 0\\ 0 & \sqrt{18}\\ \end{bmatrix} \begin{bmatrix} 1/\sqrt2 & 1/\sqrt2\\ -1/\sqrt2 & 1/\sqrt2\\ \end{bmatrix} = \begin{bmatrix} 4 & 4\\ -3 & 3\\ \end{bmatrix} = A$$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .