17
$\begingroup$

The SVD stands for Singular Value Decomposition. After decomposing a data matrix $\mathbf X$ using SVD, it results in three matrices, two matrices with the singular vectors $\mathbf U$ and $\mathbf V$, and one singular value matrix whose diagonal elements are the singular values. But I want to know why those values are named as singular values. Is there any connection between a singular matrix and these singular values?

$\endgroup$
5

3 Answers 3

12
$\begingroup$

From On the Early History of the Singular Value Decomposition by Pete Stewart:

The term "singular value" seems to have come from the literature on integral equations. A little after the appearance of Schmidt's paper, Bateman refers to numbers that are essentially the reciprocals of the eigenvalues of the kernel as singular values. Picard combined Schmidt's results with Riesz's theorem on the strong convergence of generalized Fourier series to establish a necessary and sufficient condition for the existence of solutions of integral equations.

In a later paper on the same subject, he notes that for symmetric kernels Schmidt's eigenvalues are real and in this case (but not in general) he calls them singular values. By 1937, Smithies was referring to singular values of an integral equation in our modern sense of the word. Even at this point, usage had not stabilized. In 1949, Weyl speaks of the "two kinds of eigenvalues of a linear transformation," and in a 1969 translation of a 1965 Russian treatise on nonselfadjoint operators Gohberg and Krein refer to the "s-numbers" of an operator.

See the paper for more fascinating accounts on how SVD came to be, even before the seminal paper of Golub/Kahan.

$\endgroup$
2
  • $\begingroup$ I have long forgotten the term "s-number". Your answer makes me feel nostalgic! $\endgroup$
    – user1551
    Apr 13, 2013 at 13:53
  • $\begingroup$ Why, thank you. Reading Pete Stewart's paper made me feel fuzzy, too. On the other hand, nowadays I'm doing way more linear algebra and way less operator theory... $\endgroup$ Apr 13, 2013 at 14:05
2
$\begingroup$

From Schwartzman's The Words of Mathematics:

singular (adjective), singularity (noun): from Latin singulus "separate, individual, single," from the Indo-European root sem- "one, as one." If there is just a single example of something, that example becomes special, so singular took on the meaning "out of the ordinary." [...] The meaning "out of the ordinary, troublesome," explains why a singular matrix is a square matrix whose determinant equals $0$ rather than $1$, as the etymology implies.

Also, note that the terms singular matrix and singular value are contemporary. They made their first documented appearance in 1907 and 1908, respectively.

As such, I'd guess that singular values are called like that just because they are indeed out of the ordinary. They provide a nice invariant for any complex valued matrix.

$\endgroup$
0
$\begingroup$

"Singular" means "noninvertible," and any eigenvalue $\lambda$ of matrix $M$ makes $M - \lambda I$ a singular (that is, noninvertible) matrix.

The singular values $\sigma$ of $A$ are the values for which the matrices $$A^\top A - \sigma^2 I\\ AA^\top - \sigma^2 I$$ are singular (noninvertible).

See https://en.wikipedia.org/wiki/Singular_value#History :

This concept [of singular values] was introduced by Erhard Schmidt in 1907. Schmidt called singular values "eigenvalues" at that time. The name "singular value" was first quoted by Smithies in 1937.

Additionally (and this may simply be fortuitous coincidence), the singular vectors of $A$ are two sets of vectors, $\{v_i\}$ and $\{u_i\}$ , such that for any $v_i$, then $Av_i = \sigma_iu_i$. The image of each singular vector ($v$) is therefore a multiple of a single singluar vector ($u$)! Thus, singular vectors are vectors that map to multiples of single singular vectors.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .