4
$\begingroup$

Related to my work is the concept of Singular Value Decomposition (SVD). Namely, given some matrix $B\in\mathbb{R}^{n\times m}$, $n\geq m$, SVD can be written as $$B=U\Sigma V^T,$$ where $U\in\mathbb{R}^{n\times n}$ ($n$ left-singular vectors), $\Sigma \in\mathbb{R}^{n\times m}$ (matrix with $m$ singular values on its diagonal), and $V\in\mathbb{R}^{m\times m}$ ($m$ right-singular vectors). So, how I see it, given that there are $m$ singular values, only right-singular vectors and $m$ first left-singular vectors have the asociated singular value? To which singular value do the rest of left-singular vectors correspond to?

In case one complements $\Sigma$ to contain zero singular values such that $\Sigma\in\mathbb{R}^{n\times n}$, then $V^T$ needs also to be complemented to be a $V^T\in\mathbb{R}^{n\times m}$ matrix. But, that would imply that a vector of all zeros is also a right-singular vector. Could someone clarify this?

Furthermore, suppose that the rank of matrix $B$ is $c$, $c<m\leq n$. Does this imply that $B$ has only $d$ left- and right-singular vectors?

$\endgroup$
2
  • $\begingroup$ QUOTE: Namely, given some matrix $B\in\mathbb{R}^{n\times m}$, $n\geq m$, SVD can be written as $B=U\Sigma V^T,$ where $U\in\mathbb{R}^{n\times n}$ ($n$ left-singular vectors), $\Sigma \in\mathbb{R}^{n\times m}$ (matrix with $m$ singular values on its diagonal), and $V\in\mathbb{R}^{m\times m}$ ($m$ right-singular vectors). END OF QUOTE. I don't think that really says what the SVD is unless it is mentioned that $U$ and $V$ are orthogonal matrices and $\Sigma$ is a diagonal matrix. And one could add that the diagonal entries are nonnegative. $\endgroup$ Commented Jul 21, 2012 at 20:31
  • $\begingroup$ You have to also remember that there are two different versions of the SVD. The "thin"/"compact"/"economy" SVD is the version where the orthonormal matrix $\mathbf U$ has the same dimensions as the original matrix, and $\mathbf \Sigma$ is a square diagonal matrix. The one you are talking about in the OP is the "full" SVD, and chaohuang has already told you how the full SVD is related to the eigendecomposition of the cross-product matrix. $\endgroup$ Commented Jul 22, 2012 at 11:06

2 Answers 2

3
$\begingroup$

The left singular vectors of $B$ are eigenvectors of $BB^T$, and the right singular vectors of $B$ are eigenvectors of $B^TB$, so you can easily see why there are $n$ left singular vectors, and $m$ right singular vectors.

If $n \ge m$, then there are only $m$ singular values, which are the square root of eigenvalues of $BB^T$. If $m \ge n$, then there are only $n$ singular values, which are the square root of eigenvalues of $B^TB$. So the rest of left singular vectors do not correspond to singular values, there are the eigenvectors for the 0 eigenvalue of $BB^T$.

We always have $\Sigma \in\mathbb{R}^{n\times m}$, not $\Sigma \in\mathbb{R}^{n\times n}$ if $n > m$. Zero vectors are not singular vectors.

There are always $n$ left singular vectors, and $m$ right singular vectors.

$\endgroup$
6
  • $\begingroup$ Take $B=[2 ~~3; 4~~ 5; 2~~ 3]$ (semicolon separates rows), and perform SVD. There are 2 left- and 2 right-singular vectors. $\endgroup$
    – user506901
    Commented Jul 21, 2012 at 15:29
  • $\begingroup$ no, there are 3 left singular vectors, one of which belongs to the left null space of $B$ $\endgroup$
    – chaohuang
    Commented Jul 21, 2012 at 15:40
  • $\begingroup$ Could you be more specific (perhaps by editing your question for a more full account)? $\endgroup$
    – user506901
    Commented Jul 21, 2012 at 15:42
  • $\begingroup$ @user506901 see the edited answer. $\endgroup$
    – chaohuang
    Commented Jul 21, 2012 at 15:53
  • $\begingroup$ @chaohuand Does that mean if $B=AA^T\in\mathbb{R}^{n\times n}$, with $A\in\mathbb{R}^{n\times d}$, $d<n$, then matrix $B$ has rank $d$ (ie. it has $n-d$ zero eigenvalues)? $\endgroup$
    – user506901
    Commented Jul 23, 2012 at 12:29
0
$\begingroup$

In practice, when you want to find $U$ and $V$, you do the following,

Apply $B^{T}$ to both sides of the equqtuation $ B = U\Sigma V^{T} $. That gives you the equation $$ B^{T} B = {(U\Sigma V^{T})}^{T} (U\Sigma V^{T}) \Rightarrow V\Sigma U^{T} U\Sigma V^{T}.$$ Since $U$ is an orthogonal matrix then

$$ U^{T}U=I \implies B^{T}B = V\Sigma^2 V^T \,.$$

The last equation is nothing, but a diagonalization problem for the square matrix $B^T B$. You can do the same for $U$.

$\endgroup$
1
  • $\begingroup$ @Downvoter: What's the downvote for? $\endgroup$ Commented Aug 26, 2013 at 23:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .