# Why is the SVD named so?

The SVD stands for Singular Value Decomposition. After decomposing a data matrix $\mathbf X$ using SVD, it results in three matrices, two matrices with the singular vectors $\mathbf U$ and $\mathbf V$, and one singular value matrix whose diagonal elements are the singular values. But I want to know why those values are named as singular values. Is there any connection between a singular matrix and these singular values?

• en.wikipedia.org/wiki/Singular_value_decomposition#History Commented Apr 13, 2013 at 8:43
• Hi @arbautjc, my question was not about history of singular values. Why those values are called as singular values. Is there any proper scientific reason? or are they connected with singular matrix (non-invertible).
– S. P
Commented Apr 13, 2013 at 10:12
• "Is there any connection between singular matrix and these singular values?" - in name, no. But it is true that if the matrix has singular values equal to zero, then the matrix is singular. Commented Apr 13, 2013 at 12:53
• @S.P I know, but it's to be found in history of mathematics. Commented Apr 13, 2013 at 18:14
• Commented May 30, 2023 at 0:34

From On the Early History of the Singular Value Decomposition by Pete Stewart:

The term "singular value" seems to have come from the literature on integral equations. A little after the appearance of Schmidt's paper, Bateman refers to numbers that are essentially the reciprocals of the eigenvalues of the kernel as singular values. Picard combined Schmidt's results with Riesz's theorem on the strong convergence of generalized Fourier series to establish a necessary and sufficient condition for the existence of solutions of integral equations.

In a later paper on the same subject, he notes that for symmetric kernels Schmidt's eigenvalues are real and in this case (but not in general) he calls them singular values. By 1937, Smithies was referring to singular values of an integral equation in our modern sense of the word. Even at this point, usage had not stabilized. In 1949, Weyl speaks of the "two kinds of eigenvalues of a linear transformation," and in a 1969 translation of a 1965 Russian treatise on nonselfadjoint operators Gohberg and Krein refer to the "s-numbers" of an operator.

See the paper for more fascinating accounts on how SVD came to be, even before the seminal paper of Golub/Kahan.

• I have long forgotten the term "s-number". Your answer makes me feel nostalgic! Commented Apr 13, 2013 at 13:53
• Why, thank you. Reading Pete Stewart's paper made me feel fuzzy, too. On the other hand, nowadays I'm doing way more linear algebra and way less operator theory... Commented Apr 13, 2013 at 14:05

From Schwartzman's The Words of Mathematics:

singular (adjective), singularity (noun): from Latin singulus "separate, individual, single," from the Indo-European root sem- "one, as one." If there is just a single example of something, that example becomes special, so singular took on the meaning "out of the ordinary." [...] The meaning "out of the ordinary, troublesome," explains why a singular matrix is a square matrix whose determinant equals $0$ rather than $1$, as the etymology implies.

Also, note that the terms singular matrix and singular value are contemporary. They made their first documented appearance in 1907 and 1908, respectively.

As such, I'd guess that singular values are called like that just because they are indeed out of the ordinary. They provide a nice invariant for any complex valued matrix.

"Singular" means "noninvertible," and any eigenvalue $$\lambda$$ of matrix $$M$$ makes $$M - \lambda I$$ a singular (that is, noninvertible) matrix.

The singular values $$\sigma$$ of $$A$$ are the values for which the matrices $$A^\top A - \sigma^2 I\\ AA^\top - \sigma^2 I$$ are singular (noninvertible).

This concept [of singular values] was introduced by Erhard Schmidt in 1907. Schmidt called singular values "eigenvalues" at that time. The name "singular value" was first quoted by Smithies in 1937.

Additionally (and this may simply be fortuitous coincidence), the singular vectors of $$A$$ are two sets of vectors, $$\{v_i\}$$ and $$\{u_i\}$$ , such that for any $$v_i$$, then $$Av_i = \sigma_iu_i$$. The image of each singular vector ($$v$$) is therefore a multiple of a single singluar vector ($$u$$)! Thus, singular vectors are vectors that map to multiples of single singular vectors.