I am trying to prove some statements about singular value decomposition, but I am not sure what the difference between singular value and eigenvalue is.
Is "singular value" just another name for eigenvalue?
$\begingroup$
$\endgroup$
8
-
4$\begingroup$ They agree in finite dimensions, but not necessarily for infinite-dimensional operators. I've heard the term "singular value" applied to any value for which $(A-\lambda I)^{-1}$ either does not exist or is not continuous, while eigenvalues refer only to those values for which $(A-\lambda I)^{-1}$ does not exist. $\endgroup$– Alex BeckerApr 3, 2012 at 3:30
-
4$\begingroup$ The singular value is a nonnegative scalar of a square or rectangular matrix while an eigenvalue is a scalar (any scalar) of a square matrix. $\endgroup$– Hassan MuhammadApr 3, 2012 at 3:37
-
2$\begingroup$ My guess is that the question is about the singular value decomposition for matrices of finite-dimensional operators. $\endgroup$– yepApr 3, 2012 at 4:02
-
8$\begingroup$ They are not the same thing at all, and has nothing to do with dimension. They only agree in the special case where the matrix is symmetric. This agreement also extends (in a sense) for infinite dimensional compact operators. $\endgroup$– Nick AlgerSep 30, 2012 at 2:46
-
8$\begingroup$ @AlexBecker : The DO NOT agree in finite dimensions! Clearly you're not familiar with the singular value decomposition. All real matrices have singular values, but non-square matrices don't have eigenvalues. $\endgroup$– Michael HardyJan 23, 2013 at 4:34
|
Show 3 more comments
7 Answers
$\begingroup$
$\endgroup$
5
The singular values of a $M\times N$ matrix $X$ are the square roots of the eigenvalues of the $N\times N$ matrix $X^*\,X$ (where $^*$ stands for the transpose-conjugate matrix if it has complex coefficients, or the transpose if it has real coefficients).
Thus, if $X$ is $N\times N$ real symmetric matrix with non-negative eigenvalues, then eigenvalues and singular values coincide, but it is not generally the case!
-
7$\begingroup$ what about the case in which X is square but not symmetric. The eigenvalues of X could be negative correct? And in that case how do we define the singular values? $\endgroup$– MatteoApr 9, 2014 at 18:42
-
4$\begingroup$ @matteo: I don't understand your question. Whatever matrix $X$ you choose (square or not), the matrix $X^*X$ is Hermitian (or symmetric is the entries are real) positive definite, and the definition I provided makes sense. If $X$ is a square matrix with a negative eigenvalue, then its eigenvalues and singular values are just not the same. $\endgroup$– StudentApr 21, 2014 at 16:07
-
$\begingroup$ I guess you're right, I wasn't really thinking of the fact that they're simply different. Just to make sure about one last thing, is $X^*X$ always hermitian and positive definite? $\endgroup$– MatteoApr 21, 2014 at 16:33
-
1$\begingroup$ @Matteo: Yes, it is clearly Hermitian, but only non-negative definite, sorry (i.e. you can have zero for singular value): For every $v\in \mathbb C^n$, with $\langle .\rangle$ the usual Hermitian product on $\mathbb C^n$, you have $\langle X^*Xv,v\rangle = \langle X v,X v\rangle=\|Xv\|^2\geq 0$. $\endgroup$– StudentApr 21, 2014 at 17:13
-
1$\begingroup$ Isn't it sufficient for the square matrix to be diagonalizable, rather than symmetric (ie orthogonally diagonalizable) for the singular values to be the same as the eigenvalues? $\endgroup$– shjMay 21, 2014 at 19:12
$\begingroup$
$\endgroup$
6
Given a matrix $A$, if the eigenvalues of $A^HA$ are $\lambda_i \geq 0$, then $\sqrt{\lambda_i}$ are the singular values of $A$. If $t$ is an eigenvalue of $A$, then $|t|$ is a singular value of $A$. And here is an example should be noticed, $$A = \begin{pmatrix}1&0&1\\0&1&1\\0&0&0\end{pmatrix},$$ the eigenvalues of $A$ are $1,1,0$ while the singular values of $A$ are $\sqrt{3},1,0$.
answered Dec 14, 2013 at 13:00
-
13$\begingroup$ That's a very nice example. Another example is $$\mathbf A = \left(\begin{array}{cc}1 & 1 \\0&0\end{array}\right).$$ The eigenvalues are $1$ and $0$, the singular values are $\sqrt 2$ and $0$. $\endgroup$– hbpOct 22, 2015 at 6:18
-
12$\begingroup$ "If $t$ is an eivenvalue of A, then $|t|$ is a singular value of A" - this is not true, though it does convey part of the (admittedly vague, but still useful) intuition that the eigenvalues and singular values are "the same size" $\endgroup$ Apr 12, 2016 at 22:50
-
1$\begingroup$ @stochastic Why isn't it true? What's wrong with this argument: Let $t$ be an eigenvalue of $A$; then $\bar{t}$ is an eigenvalue of $A^*$ and hence $\bar{t}t=|t|^2$ is an eigenvalue of $A^*A$, so $|t|$ is a singular value of $A$. $\endgroup$ Jul 31, 2016 at 16:52
-
4$\begingroup$ @Hunle: no, that statement can fail even if $A$ is diagonalizable. As shown by Horn the only relationship is "Weyl's inequality" $\endgroup$– DapNov 30, 2017 at 7:04
-
1$\begingroup$ @EricKightley The problem with your argument is that if $t$ is a right eigenvalue for $A$ ($Av=tv$ for some $v$) then $\overline t$ is a left eigenvalue for $A^*$ ($\overline{Av})=\overline{v}A^*=\overline{t}\overline{v}$) $\endgroup$ Jan 31, 2018 at 15:13
$\begingroup$
$\endgroup$
4
is singular value just another name for eigenvalue?
No, singular values & eigenvalues are different.
What is the difference between Singular Value and Eigenvalue?
There are many possible answers to this question. Since I don't know what you're trying to prove, I'd recommend carefully comparing definitions between the two: eigendecomposition, singular value decomposition
[EDIT: You might find the first several chapters of the book "Numerical Linear Algebra" by Trefethen and Bau more useful than the Wikipedia article. They're available here.]
Two important points:
Notice in particular that the SVD is defined for any matrix, while the eigendecomposition is defined only for square matrices (and more specifically, normal matrices).
Notice that singular values are always real, while eigenvalues need not be real.
-
1$\begingroup$ And "normal matrix" is synonymous with "diagonalizable". $\endgroup$– makansijOct 14, 2017 at 18:54
-
4$\begingroup$ @Hunle this is so wrong. a normal matrix is unitarily similar to a diagonal matrix, while diagonalizable matrix is similar to a diagonal matrix(not necessarily unitarily). $\endgroup$– Jason HuOct 14, 2017 at 23:37
-
1
-
$\begingroup$ Actually, eigendecomposition is defined for all diagonalizable matrices, not just normal matrices. $\endgroup$– Ted HoppNov 20, 2018 at 18:48
$\begingroup$
$\endgroup$
A very clear explanation from Cleve Moler's text book https://www.mathworks.com/content/dam/mathworks/mathworks-dot-com/moler/eigs.pdf
An eigenvalue and eigenvector of a square matrix A are a scalar λ and a nonzero vector x so that $$Ax = λx.$$ A singular value and pair of singular vectors of a square or rectangular matrix A are a nonnegative scalar σ and two nonzero vectors u and v so that
$$Av = σu,$$ $$A^Hu = σv.$$
Eigenvectors and singular vectors are them same if $A$ is a real symmetric matrix (so $A^H = A$).
$\begingroup$
$\endgroup$
Singular values of the SVD decomposition of the matrix A is the square root of the eigenvalues of the matrix ($A$ multiplied by $A^T$) or ($A^T$ multiplied by $A$), the two are identical with positive eigenvalues.
$\begingroup$
$\endgroup$
Consider the comparison between largest singular value and largest eigen value of a matrix with real entries, $A\in \mathbb{R}^{m \times n}$:
\begin{align*} \sigma_{\max}(A)&= \sup_{x \in S^{n-1}} ||Ax||_2 =\sup_{x \in S^{n-1}} \sqrt{||Ax||_2^2}= \sup_{x \in S^{n-1}} \sqrt{x^\top A^\top A x} = \sqrt{\lambda_{\max}(A^\top A)} \end{align*}
For $A^\top =A$, we have, $\sigma_{\max}(A)=\sqrt{\lambda_{\max}(A^2)}=\sqrt{\lambda^2_{\max}(A)}=|\lambda_{\max}(A)|=\sup\limits_{x \in S^{n-1}}| x^\top A x|$
$\begingroup$
$\endgroup$
The eigenvalues $\mathbf{\Lambda}$ of the sample matrix $\mathbf{X}$ can be obtained applying SVD to the covariance of $\mathbf{X}$:
$$\mathbf{S} = cov(\mathbf{X}) \mapsto \mathbf{S}=\mathbf{U}\mathbf{\Lambda}\mathbf{V}^{T} $$
