if $A\in \mathbb{R}^{m \times n}$, then SVD of A is $U\Sigma V^T$. I've seen different version of SVD. In the first one, $U\in \mathbb{R}^{m \times m}$, $\Sigma \in \mathbb{R}^{m\times n}$, and $V \in \mathbb{R}^{n\times n}$. In the second one $U\in \mathbb{R}^{m \times r}$, $\Sigma \in \mathbb{R}^{r\times r}$, and $V \in \mathbb{R}^{r\times n}$, where $r$ is the rank of $A$. Is there difference in these two forms or does one of them have an advantage over the other one? For instance when $U$ and $V$ are square matrices, then they are invertible, i.e, $U^TU=UU^T=I$. but when $U$ is not square only $U^TU=I$ is true.

Second question. if $A=U\Sigma V^T$, I've seen that $U^TU=I$. Can we deduce that $UU^T=I$, too?


First question

It follows from the definition of the SVD. The first one

$$ A_{m \times n} = U_{m \times m} \Sigma_{m \times n} V_{n \times n}^{T} \tag{1} $$

is the full SVD

the second one

$$ A_{m \times n} = U_{m \times r} \Sigma_{r \times r} V_{r \times n}^{T} \tag{2} $$

is the truncated SVD or reduced SVD where $A$ has rank $r$

visually we see the difference

enter image description here

enter image description here

Second question

It follows from the definition of the SVD. $U$ is orthogonal, which means that

$$ U^{T}U = UU^{T} = I \tag{3} $$


In the SVD

$$ \sigma_{1} \geq \sigma_{2} \geq \cdots \sigma_{r} \geq 0 \tag{4} $$

if you multiply

$$ U_{m \times m} \cdot \Sigma_{m \times m} V_{m \times n}^{T}\tag{5} $$

and there are only $r$ singular values then it is $0$ after that. A diagonal matrix simply multiplies across. So we only care about

$$ U_{m \times r}\Sigma_{r \times r} V_{r \times n}^{T} \tag{6} $$

  • $\begingroup$ But if we write $U=[U_1 U_2]$, where $U_1 \in \mathbb{R}^{m\times r}$ and $U_2 \in \mathbb{R}^{m\times (m-r)}$, then $I_{m\times m}=UU^T=U_1U_1^T+U_2U_2^T$. Why $U_2U_2^T=0$? $\endgroup$ – S_Alex Oct 14 '18 at 0:33
  • $\begingroup$ math.stackexchange.com/questions/1771013/null-space-from-svd $\endgroup$ – Shogun Oct 14 '18 at 0:39
  • $\begingroup$ sorry that doesn't make sense...I think $\endgroup$ – Shogun Oct 14 '18 at 0:50
  • $\begingroup$ it equals zero because the singular values are zero past the rank $r$ $\endgroup$ – Shogun Oct 14 '18 at 1:01
  • $\begingroup$ you have $U_{n} \Sigma _{n} = 0 $ $\endgroup$ – Shogun Oct 14 '18 at 1:02

Basically these two forms are equivalent. The second form can be deduced from the first form, whose elements are those of the second form and lots of zero, So when applied to data storage the second form take advantage, because you can ignore plenty of zeros and thus deal with a lower dimension. Another thing is, if U is not a square matrix then the conclusion is not true. Take (0 1) as an example. Also you may consider the rank of UU' and U'U, one of whom can't be full-rank. But there's indeed a conclusion that they share the same nonzero egienvalues, including their dimension!

  • $\begingroup$ But Ryan claims that it is true. $\endgroup$ – S_Alex Oct 14 '18 at 0:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.