3
$\begingroup$

I'm having a hard time trying to pin down why calculating the eigenvectors and eigenvalues of a matrix almost always gives something useful. There are numerous answers here on SX that discuss how to calculate them and their utility in the context of specific applications, but I haven't been able to find any answers that specifically discuss how or why these relatively simple operations produce useful answers in so many different disciplines.

For example,

  • In analysis (data science), the eigenvalues and eigenvectors of the covariance matrix give you an orthogonal basis with each eigenvector accounting for as much variance as possible and the eigenvalues tell you how much variance that eigenvector "captures" (i.e., principal components analysis, PCA). More importantly, it's a dimension reduction technique that enables you to treat high-dimensional data in an approximate way so that you might still get a useful answer out of it.
  • In quantum mechanics, the eigenvectors of an operator (represented as a matrix) give you the eigenstates of the system and the eigenvalues tell you how likely you are to measure it in that state.
  • In mechanics, the eigenvectors of the moment of inertia matrix tell you the principal axes around which an object will rotate (the eigenvalues probably give you something useful, as well, but I can't remember).
  • In social network analysis, eigenvector centrality is a way of calculating the "influence" of a node.

I have the feeling that there must be some reason why eigenvectors and eigenvalues are so broadly applicable. What is it that this operation does that almost always results in something useful coming out?

Edit, clarification: Mechanically, the procedure is always the same. Formulate a problem as having a set of inputs and generate a matrix of values that are a function of the pairwise combinations of the inputs. Then, calculating the eigenvectors and eigenvalues of this matrix will (typically) give you something useful.

What, specifically, does calculating eigenvectors and eigenvalues of this matrix do that gives you something useful?

$\endgroup$
  • 1
    $\begingroup$ See Real life examples. $\endgroup$ – Alex Silva Sep 29 '16 at 19:17
  • 2
    $\begingroup$ If you haven't seen it yet, this question seems very relevant. $\endgroup$ – Omnomnomnom Sep 29 '16 at 19:19
  • $\begingroup$ Thanks to you both for the examples, but I was hoping for something deeper. I can follow the mechanics of the examples to understand how they work and what their result is, but what I'm searching for is a unified theory of why eigenvectors and eigenvalues provide something useful across a broad range of problems. The accepted answer in the second link comes close, but is only applicable to mechanical/electrical systems. Please see Rahul's comment, below, and my replies, which try to clarify what I was looking for. $\endgroup$ – matmat Oct 4 '16 at 17:53
2
$\begingroup$

I dont know if this is the intuition you are looking for, but basically eigenvectors and diagonalization uncouple complex problems into a number of simpler problems.

In physics often stress or movement in one direction, the $x$ direction will cause a stress or movement in the $y$ and $z$ direction. An eigenvector is a direction where stress or movement in the eigendirection remain in the eigendirection, thus chosing an eigen basis replaces a complex $3$dimensional problem by three $1$-dimensional problems.

You talk about pairwise combinations of inputs, eigenvectors simplify this into function of just one input.

Look up weakly coupled oscillatory systems in a dynamics book. Systems wich complex oscillations are analysed into eigenvectors with periodic oscillations.

$\endgroup$
  • 1
    $\begingroup$ In short, eigenvectors provide the basis in which the problem becomes decoupled. +1, that's exactly it. $\endgroup$ – Rahul Sep 29 '16 at 22:17
  • $\begingroup$ Thanks Rene, I think this is a good start but that there's more to be understood. Your explanation makes sense for systems in which the behavior of interest can be captured in a linear transformation, but what about problems like PCA or the PageRank algorithm? In these problems, there's no obvious (to me) linear transformation that can be decomposed. What is it that eigenvector/eigenvalue decomposition does that makes it broadly useful not just in analyzing mechanical systems, but also in analyzing other problems that can be represented as matrices? $\endgroup$ – matmat Oct 4 '16 at 17:28
  • $\begingroup$ Rahul, I think you're closer to the mark: For both mechanical systems and PCA, eigenvector/eigenvalue decomposition breaks up the system in a way that helps to understand the problem. Can we expand on that? At the input stage, what do the problems of PCA and eigenmode analysis have in common? They both represent the problem as matrices, but is there some kind of information represented in those matrices that is common to both problems? What I'm really after, though, is how does eigenvector/eigenvalue analysis transform that information to produce something useful in both cases? $\endgroup$ – matmat Oct 4 '16 at 17:51
0
$\begingroup$

The goal with an eigenvalue decomposition is to simplify stuff so that we can understand it. Eigenvalue decompositions is a first step towards canonical transformations which allows to understand more advanced algebraic concepts by similar simplification procedures.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.