The SVD stands for Singular Value Decomposition. After decomposing a data matrix $\mathbf X$ using SVD, it results in three matrices, two matrices with the singular vectors $\mathbf U$ and $\mathbf V$, and one singular value matrix whose diagonal elements are the singular values. But I want to know why those values are named as singular values. Is there any connection between a singular matrix and these singular values?

  • $\begingroup$ en.wikipedia.org/wiki/Singular_value_decomposition#History $\endgroup$ – Jean-Claude Arbaut Apr 13 '13 at 8:43
  • $\begingroup$ Hi @arbautjc, my question was not about history of singular values. Why those values are called as singular values. Is there any proper scientific reason? or are they connected with singular matrix (non-invertible). $\endgroup$ – S. P Apr 13 '13 at 10:12
  • 1
    $\begingroup$ "Is there any connection between singular matrix and these singular values?" - in name, no. But it is true that if the matrix has singular values equal to zero, then the matrix is singular. $\endgroup$ – J. M. is a poor mathematician Apr 13 '13 at 12:53
  • $\begingroup$ @S.P I know, but it's to be found in history of mathematics. $\endgroup$ – Jean-Claude Arbaut Apr 13 '13 at 18:14

From On the Early History of the Singular Value Decomposition by Pete Stewart:

The term "singular value" seems to have come from the literature on integral equations. A little after the appearance of Schmidt's paper, Bateman refers to numbers that are essentially the reciprocals of the eigenvalues of the kernel as singular values. Picard combined Schmidt's results with Riesz's theorem on the strong convergence of generalized Fourier series to establish a necessary and sufficient condition for the existence of solutions of integral equations.

In a later paper on the same subject, he notes that for symmetric kernels Schmidt's eigenvalues are real and in this case (but not in general) he calls them singular values. By 1937, Smithies was referring to singular values of an integral equation in our modern sense of the word. Even at this point, usage had not stabilized. In 1949, Weyl speaks of the "two kinds of eigenvalues of a linear transformation," and in a 1969 translation of a 1965 Russian treatise on nonselfadjoint operators Gohberg and Krein refer to the "s-numbers" of an operator.

See the paper for more fascinating accounts on how SVD came to be, even before the seminal paper of Golub/Kahan.

  • $\begingroup$ I have long forgotten the term "s-number". Your answer makes me feel nostalgic! $\endgroup$ – user1551 Apr 13 '13 at 13:53
  • $\begingroup$ Why, thank you. Reading Pete Stewart's paper made me feel fuzzy, too. On the other hand, nowadays I'm doing way more linear algebra and way less operator theory... $\endgroup$ – J. M. is a poor mathematician Apr 13 '13 at 14:05

From Schwartzman's The Words of Mathematics:

singular (adjective), singularity (noun): from Latin singulus "separate, individual, single," from the Indo-European root sem- "one, as one." If there is just a single example of something, that example becomes special, so singular took on the meaning "out of the ordinary." [...] The meaning "out of the ordinary, troublesome," explains why a singular matrix is a square matrix whose determinant equals $0$ rather than $1$, as the etymology implies.

Also, note that the terms singular matrix and singular value are contemporary. They made their first documented appearance in 1907 and 1908, respectively.

As such, I'd guess that singular values are called like that just because they are indeed out of the ordinary. They provide a nice invariant for any complex valued matrix.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.