# Eigenvalues, singular values, and the angles between eigenvectors

Suppose the $n \times n$ matrix $A$ has eigenvalues $\lambda_1, \ldots, \lambda_n$ and singular values $\sigma_1, \ldots, \sigma_n$. It seems plausible that by comparing the singular values and eigenvalues we gets some sort of information about eigenvectors. Consider:

a. The singular values are equal to the absolute values of eigenvalues if and only if the matrix is normal, i.e., the eigenvectors are orthogonal (see http://en.wikipedia.org/wiki/Normal_matrix , item 11 of the "Equivalent definitions" section ).

b. Suppose we have two distinct eigenvalues $\lambda_1, \lambda_2$ with eigenvectors $v_1, v_2$. Suppose, hypothetically, we let $v_1$ approach $v_2$, while keeping all the other eigenvalues and eigenvectors the same. Then the largest singular value approaches infinity. This follows since $\sigma_{\rm max} = ||A||_2$ and $A$ maps the vector $v_1 - v_2$, which approaches $0$, to $\lambda_1 v_1 - \lambda_2 v_2$, which does not approach $0$.

It seems reasonable to guess that the more equal'' $|\lambda_1|, \ldots, |\lambda_n|$ and $\sigma_1, \ldots, \sigma_n$ are, the more the eigenvectors look like an orthogonal collection. So naturally my question is whether there is a formal statement to this effect.

• As written, your point a is wrong: The singular values are always $\ge0$, while the eigenvalues can be complex, even for orthogonal eigenvectors. Aug 4, 2012 at 19:31
• To restate @celtschk's comment: the singular values and eigenvalues are the same iff the matrix is symmetric positive semidefinite. Aug 4, 2012 at 19:39
• ...I forgot to take the absolute value of the eigenvalues in the first version of this question. Fixed now, I hope. Aug 4, 2012 at 20:26
• Can we assume that the eigenvalues are separated from each other by some amount, say $|\lambda_i-\lambda_j|>\delta$ for $\delta$ independent of i,j? Aug 5, 2012 at 4:37

We can gain some insight into the question if we think in terms of change of basis and the operator norm $\|A\| = \sigma_1$. I assume that we've arranged the eigenvalues so that $|\lambda_1| \geq \ldots \geq |\lambda_n| > 0$. So $A = E\Lambda E^{-1}$, with $\Lambda$ having the eigenvalues in order on the diagonal, and $E$ being the matrix whose columns are the corresponding eigenvectors. We can consider $E^{-1}$ to be the matrix that represents a coordinate transform, bringing the eigenvectors to the standard basis.

Suppose the singular values of $E$ are $s_1, \ldots, s_n$. It follows that the singular values of $E^{-1}$ are $s_n^{-1}, \ldots, s_1^{-1}$. Since $E^{-1}$ is not an isometry, the matrix norm of $A$ is not the same as $\|\Lambda\| = |\lambda_1|$. In fact we have $$\sigma_1 = \|A\| \leq \|E\| \|\Lambda\| \|E^{-1}\| = s_1 |\lambda_1| s_n^{-1}.$$ The quantity $\kappa(E) = \kappa(E^{-1}) = s_1 s_n^{-1}$ is the condition number of the change of basis matrix $E^{-1}$, and it coincides with the condition number of $E$ itself. The condition number gives a measure of how close $E$ is to being degenerate. For an orthogonal matrix the condition number is 1, but as you let two columns of $E$ "approach" each other, the condition number of $E$ increases. If you think of the columns of $E$ as forming the sides of a box (a parallelotope), then the condition number is a measure of how squashed the box is (this assertion could be made precise).

Anyway, you have that $$\kappa(E)^{-1}|\lambda_1| \leq \sigma_1 \leq \kappa(E) |\lambda_1|,$$ which is at least a partial answer to your question.

A little help in understanding the relation or difference between eigenvectors and singular vectors is elaborated in detail in: SVD vs EVD The bookmark leads you to a comparison between evd and svd. But there's much more there if you browse through the pages.

EngleField and Farr show an elegant derivation for the angle between eigenvectors:

Englefield, M. J., & Farr, G. E. (2006). Eigencircles of 2 x 2 Matrices. Mathematics Magazine Vol. 79 Oct.,2006, 281-289. on page 285

The angle between the singular vectors is the angle of the rotation $$UV^T$$ where $$U$$ and $$V$$ contain the singular vectors of $$A = U \Sigma V^T$$

Eigencircle representation leads you to a figure showing this angle on an eigencircle plot. • Can you tell me why this is voted down? Then I can improve it, downvoting without a comment does not help me to imrove. Nov 30, 2019 at 12:49