Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
There are already good answers about importance of eigenvalues / eigenvectors, such as this question and some others, as well as this Wikipedia article.
I know the theory and these examples, but now in order to do my best to prepare a course I'm teaching, I'm looking for ideas about good real life examples of usage of these concepts.
Do you know some good simple real-life examples (in economics or data analysis or anything else), in which the usage of eigen values/vectors is a crucial tool?
Here are just some of the many uses of eigenvectors and eigenvalues:
Using singular value decomposition for image compression. This is a note explaining how you can compress and image by throwing away the small eigenvalues of $AA^T$. It takes an $8$ megapixel image of an Allosaurus, and shows how the image looks after compressing by selecting $1$,$10$,$25$,$50$,$100$ and $200$ of the largest singular values.
Deriving Special Relativity is more natural in the language of linear algebra. In fact, Einstein's second postulate really states that "Light is an eigenvector of the Lorentz transform." This document goes over the full derivation in detail.
Spectral Clustering. Whether it's in plants and biology, medical imaging, buisness and marketing, understanding the connections between fields on Facebook, or even criminology, clustering is an extremely important part of modern data analysis. It allows people to find important subsystems or patterns inside noisy data sets. One such method is spectral clustering which uses the eigenvalues of a the graph of a network. Even the eigenvector of the second smallest eigenvalue of the Laplacian matrix allows us to find the two largest clusters in a network.
Dimensionality Reduction/PCA. The principal components correspond the the largest eigenvalues of $A^TA$ and this yields the least squared projection onto a smaller dimensional hyperplane, and the eigenvectors become the axes of the hyperplane. Dimensionality reduction is extremely useful in machine learning and data analysis as it allows one to understand where most of the variation in the data comes from.
Low rank factorization for collaborative prediction. This what Netflix does (or once did) to predict what rating you'll have for a movie you have not yet watched. It uses the SVD, and throws away the smallest eigenvalues of $A^TA$.
The Google Page Rank algorithm. The largest eigenvector of the graph of the internet is how the pages are ranked.
In control theory and dynamical systems you have modal decomposition, which is a very useful tool to quickly create the dynamic equation for a given (real life) system
Given a system of differential equation:
$\dot x(t) = Ax(t)$, $x(0) = x_o$, $A$ has distinct eigenvalues
Then the solution to this equation is given as:
$x(t) = \sum\limits_{i=1}^n c_ie^{\lambda_it}v_i$
where $c_i$ are the coefficient corresponding to initial condition $x(0)$, $v_i$ is the $i$th eigenvector, and $\lambda_i$ is the $i$th eigenvalue, needless to say $v_i, \lambda_i$ forms a pair
The physical interpretation is that the solution corresponds to the unforced/natural response of the system and is used to analyze bridge models, RC circuits, mass-spring-damper, magnetic suspension, fluid dynamics, acoustics, neuron models...
Further, we can look at the eigenvalue of the $A$ matrix to determine the stability of the system. If all eigenvalues lie in the open left half plane, then the matrix $A$ is known simply as Hurwitz (a linear algebra result completely detached from dynamical system), and the system is asymptotically stable. Otherwise it will either have a state that never goes to zero, or blow up as time goes to infinity.
This result is extremely well known, but goes by different names, in some field this is simply known as the eigenvector-eigenvalue problem: http://jupiter.math.nctu.edu.tw/~tshieh/teaching/Math254_summerI2009/MAth254_summer_note/lecture16.pdf http://tutorial.math.lamar.edu/Classes/DE/RealEigenvalues.aspx https://see.stanford.edu/materials/lsoeldsee263/11-eig.pdf
You can also consult basic references on ODE, such as Boyce and DiPrima
In real life, we effectively use eigen vectors and eigen values on a daily basis though sub-consciously most of the time.
Example 1: When you watch a movie on screen(TV/movie theater,..), though the picture(s)/movie you see is actually 2D, you do not lose much information from the 3D real world it is capturing. That is because the principal eigen vector is more towards 2D plane the picture is being captured and any small loss of information(depth) is inferred automatically by our brain. (reason why we most of the times take photos using camera facing directly at us, not from the top of the head). Each scene requires certain aspects of the image to be enhanced, that is the reason the camera man/woman chooses his/her camera angle to capture most of those visual aspects. (apart from colour of costume, background scene and background music)
Example 2: If you eat pizza, french fries,...or any food.... you are typically translating their taste into sour, sweet, bitter,salty, hot, etc ... principal components of taste -- though in reality the way a food is prepared is formulated in terms of ingredients' ratios (sugar,flour,butter, etc...10's of 100's of things that go into making a specific food) ... However our mind will transform all such information into the principal components of taste(eigen vector having sour, bitter, sweet, hot,..) automatically along with the food texture and smell. So we use eigen vectors every day in many situations without realizing that's how we learn about a system more effectively. Our brain simply transforms all the ingredients, cooking methods, final food product into some very effective eigen vector whose elements are taste sub parts ,smell and visual appearance internally. (All the ingredients and their quantities along with the cooking procedure represent some transformation matrix A and we can find some principal eigen vector(s) V with elements as taste+smell+appearance+touch having some linear transformation directly related. AV = wV , where w represent eigen values scalar and V an eigen vector) (top wine tasters probably have bigger taste+smell+appearance eigen vector and also with much bigger eigen values in each dimension. This concept can be extended to any field of study.)
Example 3: if we take pictures of a person from many angles(front , back, top, side..) on a daily basis and would like to measure the changes in the entire body as one grows,... we can get the most information from the front angle with the axis of camera perpendicular to the line passing from crown of the head to a point passing between one's feet. This axis/camera angle captures the most useful information to measure a person's outer physical body changes as the age progresses. This axis becomes a principal eigen vectors with the largest eigen values. (Note: the data/images that we capture directly from the top of the person may give very less useful information compared to the camera directly facing him/her in this situation. That is the reason why we use PCA-Princial Component Analysis technique in determining most effective eigen vectors and related eigen values to capture most of the needed information without bothering about all the remaining axes of data capture.)
Hope this helps in understanding why and how we use eigen vectors and eigen values for better perception in whatever we do on a day to day . Eigen vectors represent those axes of perception/learn along which we can know/understand/perceive things around us in very effective way(s).
Finally it boils down to the differences between person to person, in consciously/sub-consciously building/refining such principal eigen vectors and related eigen values, in each field of learning that differentiate one person from the other. ( ex: musicians, artists, scientists, mathematicians, camera men, directors, teachers, doctors, engineers, parents, stock market brokers, weather prediction, ....)
Eigenvectors are axes, eigenvalues are distances along these axes
Eigenvectors are the base of dimensional reduction techniques like PCA (principal component analysis), extremely useful in situations where we want to reduce the number of dimensions to a more practical one.
Concrete example: We want to find similar pictures in a large set and show the relationships in 2D (we don't know what are the similarity criteria, not either their number):
The result above is obtained using a non guided simple dimension reduction technique. Probably it isn't very useful, but it illustrates the creation of partial clusters, e.g. the direction the person is looking to, or the color of the skin, or whether the person opens the mouth, or who the person is. While there are plenty of criteria measured, we are able to use a 2D system to reveal their effects combined, this is actually the expected benefit. The algorithm here found two super abstract criteria which are eigenvectors, and returned the corresponding pair of eigenvalues for each picture, used as individual coordinates to arrange the set.
Face features as eigenvector: Eigenface
Using eigenvectors is a base technique in face recognition where we want to associate a name to a person picture. The eigenvectors in this case are eigenfaces.
Imagine we got black and white images of 47x62 pixels which can have some gray attribute, we actually have data with a value in 1348 dimensions:
Not all pixels are critical in a given image. We want to reduce the number of dimensions to the "useful" ones, without losing the main features of the images. Say we want to move from 1348 to 150 dimensions. This is the usual way of pre-processing images before doing some image classification, like face recognition, in order to decrease CPU workload.
The reduction is done by finding eigenvectors of the input images, these eigenvectors can be seen as basis images, from which the complete (actually nearly complete) images can be reconstructed.
Below are the 32 first eigenvectors, out of 150 which were computed by some PCA, in order of usefulness, that is they are dimensions along which original images have the highest variance. These images are somehow like the major harmonics of a sound (obtained using a Fourier transform):
Note the three first eigenvectors are luminosity related, the main variance in images not shot in controlled conditions.
To reconstruct the images from these eigenvectors, we only need to know the associated eigenvalues. The information to store and/or process for each image is now a vector of 150 eigenvalues, instead of the original vector of 1348 pixel values. A large gain. Still not much information has been lost. Here is a subset of original images, and images reconstructed using the eigenvectors shown above:
The number of eigenvectors used (and their choice) determines how much variance from the pixels is lost. This number is labeled below as "number of components", from the name of the reduction technique used: The principal component analysis (PCA):
For this graph, we see:
This is somehow similar to image compression, but much more too, because the eigenvalues can be used directly to make a relation between the image and the name of the person, we just need to train the classifier with a set of known images associated with the correct category (here the name to associate).
(I took my examples from this excellent book on machine learning. Faces pictures are from the LFW dataset)
eigen values can be applied many ways. their significance and the purporse depend upone your decision or accepted decision to use the eigen values how you see fit. In a real word example you have can the eigen values determind for almost any graph. you give rise to the matrix which are vectors or translations of electrical systems or mechanical movements that you then compute the eigen values and obtain the eigen vectors for those values. so you will have to then interpret the resluts or reapply the data back to the graph you started with and/or a list of the values and vectors and interpret.
so robot electrical systems are a great place to investigate these values. they help determine the systems electrical responses such as voltages and mechanical responses such as movements.
hope it helps
Let me give you a direct answer. In application eigen values can be: 1- Control Field: eigen values are the pole of the closed loop systems, if there values are negative for analogue systems then the system is stable, for digital systems if the values are inside the unit circle also the system is stable.
2- Mechanical system: eigen values are the natural frequency and the eigen values are the mode shapes.
Control Field: eigen values are the pole of the closed loop systems what is a pole in this context? what is a close loop system? In order to speak of eigen value, we need a function $f$ such that $f(x) = \lambda x$ or a matrix $A$ such that $A X = \lambda X$. What is $A$ in this context? I have a math-only background and "close loop system" is like hieroglyph for me ;) Idem for are the natural frequency and the eigen values are the mode shapes. what is the relevant linear function $f$ or matrix $A$ ?
$\endgroup$