# A number of SVD components; understanding the relation

Related to my work is the concept of Singular Value Decomposition (SVD). Namely, given some matrix $B\in\mathbb{R}^{n\times m}$, $n\geq m$, SVD can be written as $$B=U\Sigma V^T,$$ where $U\in\mathbb{R}^{n\times n}$ ($n$ left-singular vectors), $\Sigma \in\mathbb{R}^{n\times m}$ (matrix with $m$ singular values on its diagonal), and $V\in\mathbb{R}^{m\times m}$ ($m$ right-singular vectors). So, how I see it, given that there are $m$ singular values, only right-singular vectors and $m$ first left-singular vectors have the asociated singular value? To which singular value do the rest of left-singular vectors correspond to?

In case one complements $\Sigma$ to contain zero singular values such that $\Sigma\in\mathbb{R}^{n\times n}$, then $V^T$ needs also to be complemented to be a $V^T\in\mathbb{R}^{n\times m}$ matrix. But, that would imply that a vector of all zeros is also a right-singular vector. Could someone clarify this?

Furthermore, suppose that the rank of matrix $B$ is $c$, $c<m\leq n$. Does this imply that $B$ has only $d$ left- and right-singular vectors?

-
QUOTE: Namely, given some matrix $B\in\mathbb{R}^{n\times m}$, $n\geq m$, SVD can be written as $B=U\Sigma V^T,$ where $U\in\mathbb{R}^{n\times n}$ ($n$ left-singular vectors), $\Sigma \in\mathbb{R}^{n\times m}$ (matrix with $m$ singular values on its diagonal), and $V\in\mathbb{R}^{m\times m}$ ($m$ right-singular vectors). END OF QUOTE. I don't think that really says what the SVD is unless it is mentioned that $U$ and $V$ are orthogonal matrices and $\Sigma$ is a diagonal matrix. And one could add that the diagonal entries are nonnegative. –  Michael Hardy Jul 21 '12 at 20:31
You have to also remember that there are two different versions of the SVD. The "thin"/"compact"/"economy" SVD is the version where the orthonormal matrix $\mathbf U$ has the same dimensions as the original matrix, and $\mathbf \Sigma$ is a square diagonal matrix. The one you are talking about in the OP is the "full" SVD, and chaohuang has already told you how the full SVD is related to the eigendecomposition of the cross-product matrix. –  Ｊ. Ｍ. Jul 22 '12 at 11:06

The left singular vectors of $B$ are eigenvectors of $BB^T$, and the right singular vectors of $B$ are eigenvectors of $B^TB$, so you can easily see why there are $n$ left singular vectors, and $m$ right singular vectors.

If $n \ge m$, then there are only $m$ singular values, which are the square root of eigenvalues of $BB^T$. If $m \ge n$, then there are only $n$ singular values, which are the square root of eigenvalues of $B^TB$. So the rest of left singular vectors do not correspond to singular values, there are the eigenvectors for the 0 eigenvalue of $BB^T$.

We always have $\Sigma \in\mathbb{R}^{n\times m}$, not $\Sigma \in\mathbb{R}^{n\times n}$ if $n > m$. Zero vectors are not singular vectors.

There are always $n$ left singular vectors, and $m$ right singular vectors.

-
Take $B=[2 ~~3; 4~~ 5; 2~~ 3]$ (semicolon separates rows), and perform SVD. There are 2 left- and 2 right-singular vectors. –  user506901 Jul 21 '12 at 15:29
no, there are 3 left singular vectors, one of which belongs to the left null space of $B$ –  chaohuang Jul 21 '12 at 15:40
Could you be more specific (perhaps by editing your question for a more full account)? –  user506901 Jul 21 '12 at 15:42
@user506901 see the edited answer. –  chaohuang Jul 21 '12 at 15:53
@chaohuand Does that mean if $B=AA^T\in\mathbb{R}^{n\times n}$, with $A\in\mathbb{R}^{n\times d}$, $d<n$, then matrix $B$ has rank $d$ (ie. it has $n-d$ zero eigenvalues)? –  user506901 Jul 23 '12 at 12:29

In practice, when you want to find $U$ and $V$, you do the following,

Apply $B^{T}$ to both sides of the equqtuation $B = U\Sigma V^{T}$. That gives you the equation $$B^{T} B = {(U\Sigma V^{T})}^{T} (U\Sigma V^{T}) \Rightarrow V\Sigma U^{T} U\Sigma V^{T}.$$ Since $U$ is an orthogonal matrix then

$$U^{T}U=I \implies B^{T}B = V\Sigma^2 V^T \,.$$

The last equation is nothing, but a diagonalization problem for the square matrix $B^T B$. You can do the same for $U$.

-
@Downvoter: What's the downvote for? –  Mhenni Benghorbal Aug 26 '13 at 23:40