I'm having a hard time trying to pin down why calculating the eigenvectors and eigenvalues of a matrix almost always gives something useful. There are numerous answers here on SX that discuss how to calculate them and their utility in the context of specific applications, but I haven't been able to find any answers that specifically discuss how or why these relatively simple operations produce useful answers in so many different disciplines.
- In analysis (data science), the eigenvalues and eigenvectors of the covariance matrix give you an orthogonal basis with each eigenvector accounting for as much variance as possible and the eigenvalues tell you how much variance that eigenvector "captures" (i.e., principal components analysis, PCA). More importantly, it's a dimension reduction technique that enables you to treat high-dimensional data in an approximate way so that you might still get a useful answer out of it.
- In quantum mechanics, the eigenvectors of an operator (represented as a matrix) give you the eigenstates of the system and the eigenvalues tell you how likely you are to measure it in that state.
- In mechanics, the eigenvectors of the moment of inertia matrix tell you the principal axes around which an object will rotate (the eigenvalues probably give you something useful, as well, but I can't remember).
- In social network analysis, eigenvector centrality is a way of calculating the "influence" of a node.
I have the feeling that there must be some reason why eigenvectors and eigenvalues are so broadly applicable. What is it that this operation does that almost always results in something useful coming out?
Edit, clarification: Mechanically, the procedure is always the same. Formulate a problem as having a set of inputs and generate a matrix of values that are a function of the pairwise combinations of the inputs. Then, calculating the eigenvectors and eigenvalues of this matrix will (typically) give you something useful.
What, specifically, does calculating eigenvectors and eigenvalues of this matrix do that gives you something useful?