In singular value decomposition (SVD), $X=USV^T$, if $X$ is $N\times D$, then $U$ is $N\times N$, $S$ is $N\times D$, and $V$ is $D\times D$.

Let's assume $N<D$.

Every column vector in $V$ is an eigenvector of $X^TX$, so we have $D$ eigenvectors in total.

Each (squared) diagonal element of $S$ is an eigenvalue of $X^TX$, so we have $\min(N, D)=N$ eigenvalues.

Here comes my question: why do we have unequal numbers of eigenvalues and eigenvectors? Shouldn't each eigenvector be associated with one eigenvalue?


$X^TX$ is a $D\times D$ matrix, hence it has $D$ eigenvalues (taken with multiplicity). Same goes for $S^TS$.

Each element on the diagonal of $S^TS$ is an eigenvalue of $X^TX$; moreover, the corresponding eigenvectors are columns of $V$; there are $D$ such columns, so everything fits.

  • 1
    $\begingroup$ I see. So I can't compute $X^TX$'s eigenvalues as diag(S)^2. Instead, I have to do the matrix multiplication first and then take the diagonal elements: diag(S'*S). $\endgroup$ – Sibbs Gambling Apr 23 '15 at 11:32
  • 1
    $\begingroup$ @SibbsGambling exactly. $\endgroup$ – TZakrevskiy Apr 23 '15 at 11:33
  • $\begingroup$ Oh, and actually they are all 0. :-) $\endgroup$ – Sibbs Gambling Apr 23 '15 at 11:37
  • $\begingroup$ @SibbsGambling that's strange, because if all eigenvalues of a symmetric matrix are zero, then the matrix itself is zero. And if $X^TX=0$, then $X=0$. $\endgroup$ – TZakrevskiy Apr 23 '15 at 11:41
  • 1
    $\begingroup$ @SibbsGambling no problem. The last $D-N$ eigenvalues have to be zeros, because $D> N\ge rank(X) = rank(X^TX)=rank(S^TS)$. $\endgroup$ – TZakrevskiy Apr 23 '15 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.