Why are Eigenvalues So Important? [duplicate]

Question

I understand the definition of eigenvalues and eigenvectors. Eigenvectors just stretch when applied by a linear transformation (a matrix). So far so good.

I am surprised by the amount of times they are coming up in my studies of matrix algebra and optimization. Yet, nowhere I find an intuitive explanation of why they are so important and why they keep coming up in so many problems?

Are there any eigenvalue theorems or something (like the normal distribution's explanation using the central limit theorem justifies its omnipresence) that explain why they are so important, what do they tell us?

EDIT: Please explain in terms of matrices, what do eigen values tell us about, say Hessian for example. I am trying to understand what do eigen vectors tell us about optimization problems in high dimensional spaces, Shapes of the curves, number of saddle points.

EDIT: The question in the duplicate link tallks about the connection between systems of differential equations and eigenvectors. I am trying for optimization in high dimensional spaces, Hessian based explanation.

Answer 1

Let me give you an example of eigenvectors appearing from nowhere. Let $X(t)\in\mathbb{R}^n$ satisfy the system of differential equations $\frac{dX}{dt}=CX(t)$ for some matrix $C$. Let $v$ be a left-eigenvector of $C$ with eigenvalue $\lambda$. Then $\frac{d}{dt}v\cdot X(t)=\lambda v\cdot X(t)$. From this you can show that $v\cdot X(t)=v\cdot X(0)e^{\lambda t}$ and as this is true for all eigenvectors, if your eigenvectors are linearly independent this can lead to a closed form solution for your $X_i(t)$.