4
$\begingroup$

Truncated SVD: http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.TruncatedSVD.html

Reduced SVD, I thought this is essentially the same thing, and it appears to be actually more commonly called this way.

If you could provide reference, that'll be great.

$\endgroup$
  • $\begingroup$ People can define things as they wish, but to me the former is still a decomposition of A, just without explicitly including the zero singular values. While the latter would be understood as an approximation to me. $\endgroup$ – Ian Jan 29 '18 at 19:24
  • $\begingroup$ What do you mean by approximation, I thought reduced SVD is the one without explicitly including the zero singular values, thus called reduced. I am uncertain what truncated means yet. $\endgroup$ – zyxue Jan 29 '18 at 20:03
  • $\begingroup$ @Ian, do you agree with my answer, please? Thanks! $\endgroup$ – zyxue Jan 29 '18 at 21:33
  • $\begingroup$ For the record, this video discussed this topic (full SVD vs reduced SVD vs truncated SVD): youtu.be/AbB-w77yxD0?t=81. $\endgroup$ – zyxue Jan 10 at 22:37
3
$\begingroup$

I labeled the shape under each matrix.

Suppose the shape of $A$ is $m \times n$, thus it is written as $\underset{m \times n}{A}$, and $m \ge n$.

Reduced SVD:

\begin{align} \underset{m\times n}{A_r} &= \underset{m \times n,}{U_r} \underset{n \times n,}{\Sigma} \underset{n \times n}{V_r^{T}} \end{align}

Here, the $U$ is not a square matrix. By appending addition $m-n$ orthogonal columns to $U$, and zero-vector rows to $\Sigma$, we get the full SVD:

\begin{align} \underset{m\times n}{A} &= \underset{m \times m,}{U} \underset{m \times n,}{\Sigma} \underset{n \times n}{V^{T}} \end{align}

Here is an illustration of the reduced-to-full-SVD transition from the web. The dashed box areas highlight the appended columns and rows

As for truncated SVD, we take $k$ largest singular values ($0 \lt k \lt n$, thus truncated) and their corresponding left and right singular vectors,

\begin{align} \underset{m\times n}{A} &\approx \underset{m \times k,}{U_t} \underset{k \times k,}{\Sigma} \underset{k \times n}{V_t^{T}} \end{align}

$A$ constructed via truncated SVD is an approximation to the original A.

$\endgroup$
  • 2
    $\begingroup$ You should not append zeros to U,V. You should extend them to be orthogonal matrices and put zeros in $\Sigma$. $\endgroup$ – Ian Jan 29 '18 at 22:30
  • $\begingroup$ I've made the correction. Thanks for pointing out, but I wonder why is that useful? $\endgroup$ – zyxue Jan 29 '18 at 22:51
  • 2
    $\begingroup$ It's just for theoretical reasons, so that $U,V$ are "bona fide" orthogonal matrices, which is one of the main advantages of the full SVD over the reduced one. Pretty much any real world numerical computation will be on the reduced SVD or a truncated SVD. $\endgroup$ – Ian Jan 29 '18 at 22:52
  • 1
    $\begingroup$ I think it would be more appropriate to write $A \approx U \Sigma V^T$, since the truncated SVD is just an approximation and you said that yourself. $\endgroup$ – Integral Jun 30 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.