This question already has an answer here:

I understand the definition of eigenvalues and eigenvectors. Eigenvectors just stretch when applied by a linear transformation (a matrix). So far so good.

I am surprised by the amount of times they are coming up in my studies of matrix algebra and optimization. Yet, nowhere I find an intuitive explanation of why they are so important and why they keep coming up in so many problems?

Are there any eigenvalue theorems or something (like the normal distribution's explanation using the central limit theorem justifies its omnipresence) that explain why they are so important, what do they tell us?

EDIT: Please explain in terms of matrices, what do eigen values tell us about, say Hessian for example. I am trying to understand what do eigen vectors tell us about optimization problems in high dimensional spaces, Shapes of the curves, number of saddle points.

EDIT: The question in the duplicate link tallks about the connection between systems of differential equations and eigenvectors. I am trying for optimization in high dimensional spaces, Hessian based explanation.


marked as duplicate by skyking, Zev Chonoles, David K, JonMark Perry, daw Feb 23 '17 at 7:20

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 1
    $\begingroup$ Also: math.stackexchange.com/questions/357340/…, $\endgroup$ – Moo Feb 23 '17 at 6:10
  • 1
    $\begingroup$ The answer is because in the right basis every matrix looks like a very nice upper triangular matrix with eigenvalues on the diagonal. axler.net/DwD.pdf does a very good job of walking you through the relevant results in an intuitive way. $\endgroup$ – btilly Feb 23 '17 at 6:14

Let me give you an example of eigenvectors appearing from nowhere. Let $X(t)\in\mathbb{R}^n$ satisfy the system of differential equations $\frac{dX}{dt}=CX(t)$ for some matrix $C$. Let $v$ be a left-eigenvector of $C$ with eigenvalue $\lambda$. Then $\frac{d}{dt}v\cdot X(t)=\lambda v\cdot X(t)$. From this you can show that $v\cdot X(t)=v\cdot X(0)e^{\lambda t}$ and as this is true for all eigenvectors, if your eigenvectors are linearly independent this can lead to a closed form solution for your $X_i(t)$.

  • $\begingroup$ thanks, please see my edits in the question. $\endgroup$ – Rafael Feb 23 '17 at 7:05

Not the answer you're looking for? Browse other questions tagged or ask your own question.