# How are eigenvectors/eigenvalues and differential equations connected?

In school and at university we never had eigenvalues nor differential equations, so these concepts were really giving me a hard time. Now I developed some intuition for both concepts.

I learned that both are connected in some way, i.e. there is an eigenvector/eigenvalue approach to solve differential equations. Unfortunately most of the text I find are way beyond my understanding and seem to require very extensive study.

Is there a possibility to provide me with an intuition of the connection anyway and perhaps give an easy example of how to use this approach to solve a differential equation?

-
Suggest replacing "differential equations" by "linear differential equations" in the title. If the equation is not linear, linearizable, approximately linear, asymptotically linear, or otherwise close to the linear world, it is not as likely to have a connection to linear algebra. –  T.. Nov 15 '10 at 20:42
I have some lecture notes on linear algebra math.princeton.edu/~wwong/teach.html#202S2008 on-line. Unfortunately linear differential equations are only dealt with at the last week (which means that you would have to read through all 12 weeks of the notes to get a picture). If you have some background, you can try starting in week 9 or 10 where discrete dynamical systems are first discussed. –  Willie Wong Nov 15 '10 at 21:04

You could think of eigenvectors as giving you the coordinate system in which your problem has the simplest form. It's analogous to working physical problems by turning your coordinate system the right way to make some term go away.

-

If you have a linear system on matrix form, $dX/dt=AX$, where $X(t)$ is a vector in $\mathbf{R}^n$ and $A$ is an $n\times n$ constant real matrix, then $X(t) = \exp(\lambda t) V$ is a solution to the system if $V$ is an eigenvector of $A$ with eigenvalue $\lambda$. (This works since $\exp(\lambda t)$ is an eigenfunction of the differential operator $d/dt$ with eigenvalue $\lambda$.)

If $A$ is diagonalizable, then the general solution is a linear combination of terms like above: $X(t)=\sum_{k=1}^n c_k \exp(\lambda_k t) V_k$. (If the eigenvalues are complex, this is the general complex-valued solution; one can perform some standard tricks to extract the general real-valued solution from that.) Nondiagonalizable matrices $A$ cause a bit more trouble, but that can be handled too.

(Of course, this is related to what has been said in other answers already, but with a system of this type one sees clearly that the eigenvectors play a role too, not just the eigenvalues.)

-

Suppose $\sum a_n y^{(n)} = 0$ is a homogeneous linear differential equation with constant coefficients. Let $D$ be the linear operator defined on, say, the space $C^{\infty}(\mathbb{R})$ of smooth functions $\mathbb{R} \to \mathbb{R}$ which sends a function $y$ to its derivative $y'$. Then the solutions to the above differential equation are the nullspace of the operator $\sum a_n D^n$.

It turns out that this nullspace is finite-dimensional, so we can use finite-dimensional linear algebra on it. It is not just a vector space, though: it comes equipped with an action of $D$, which commutes with $\sum a_n D^n$, and which therefore splits the nullspace into eigenspaces of $D$. In the simplest case, these eigenspaces will all be one-dimensional. Now, the eigenvectors of $D$ are precisely the functions satisfying $Dy = \lambda y$ for some eigenvector $\lambda$, which are precisely the exponential functions $y = e^{\lambda t}$. And it's not hard to see that such a function is a solution of the differential equation if and only if $\lambda$ is a root of the characteristic polynomial $\sum a_n \lambda^n$.

Example. Simple harmonic motion is governed by the differential equation $m D^2 + k D^0 = 0$, which has characteristic polynomial $m \lambda^2 + k = 0$. The roots of this polynomial are complex, but if we allow ourselves to work with complex numbers (formally, in the above situation we tensor with $\mathbb{C}$) we find that the eigenvalues are $\lambda = \pm i \sqrt{ \frac{k}{m} }$, so the set of solutions is all functions of the form

$$x = A e^{ i \sqrt{ \frac{k}{m} } t } + B e^{-i \sqrt{ \frac{k}{m} } t}.$$

By Euler's formula, if we restrict our solutions to be real we get the familiar periodic sine and cosine.

In general the eigenspaces will not be one-dimensional and then the theory of Jordan normal form applies. This occurs, for example, when finding the general form of damped harmonic motion.

-
Shouldn't $e^{\lambda y}$ read something like $e^{\lambda t}$ instead? –  joshphysics Sep 23 '14 at 1:29
@joshphysics: yep. Thanks for the correction! –  Qiaochu Yuan Sep 23 '14 at 1:34
Great answer btw. Would you happen to know of any good references that treat ODEs heavily using linear-algebraic language as in this answer? In particular, a text which discusses Jordan normal form in this context as you allude to would be useful. I'll be teaching a math methods for physics class, and I'd find such a reference very useful. –  joshphysics Sep 27 '14 at 4:50
@joshphysics: nope, I'm not aware of a reference that does this. –  Qiaochu Yuan Sep 28 '14 at 2:21

The collection of all functions that are differentiable at least $n$ times (let's call it $\mathcal{F}_n$ just for the purposes of this post) form a vector space; and the operator $D$ defined by $D(f) = f'$ is linear ($D(f+g)=D(f)+D(g)$, and $D(\alpha f) = \alpha D(f)$ for all $f,g$ and scalars $\alpha$). So $D$ is a linear transformation from the vector space of functions that can be differentiated at least $n$ times, and functions that can be differentiated at least $n-1$ times.

Now, suppose you have a homogeneous linear differential equation: $$f_n(x) y^{(n)}(x) + \cdots + f_1(x)y'(x) + f_0(x)y(x) = 0.$$ Then the collection of all function $y(x)$ that are solutions to this system form a subspace of $\mathcal{F}_n$: because if $y_1$ and $y_2$ are solutions, then so is $y_1+\alpha y_2$ for any scalar $\alpha$, and the zero function is certainly a solution. It is, in fact, the nullspace of a certain linear transformation, namely, the linear transformation $L$ given by $$L(y) = f_n(x)y^{(n)}(x)+\cdots + f_1(x)y'(x) + f_0(x)y(x).$$ So one can bring some linear algebra to bear on this problem; e.g., determining the dimension of the solution space, etc.

Also, in analogy to the case of systems of linear equations, consider a *non*homogeneous linear differential equation: $$f_n(x) y^{(n)}(x) + \cdots + f_1(x)y'(x) + f_0(x)y(x) = g(x).$$ Suppose you could find a particular solution $y_p$ to this equation. Then, if $y_h$ is any solution to the corresponding homogeneous equation, then $y_p+y_h$ is a solution to the nonhomogeneous equation as well; and if $z_p$ is another solution to the nonhomogeneous equation, then $y_p-z_p$ is a solution to the homogeneous equation. So every solution to the nonhomogeneous equation is of the form $y_p + y_h$, where $y_p$ is the particular solution to found, and $y_h$ is a solution to the associated homogeneous equation. This is exactly the same thing that happens with systems of linear equations (where the unknowns are numbers).

In many cases, you can show that any solutions to your equations will actually be infinitely differentiable (e.g., if your $f$s are infinitely differentiable). Then you can think of the linear transformation $L$ as a linear operator on the space of infinitely differentiable functions. Then eigenvectors and eigenvalues can come into play: eigenvectors and eigenvalues give you simpler ways of thinking about a linear transformation, so they give you simpler ways of thinking about this particular linear transformation (which happens to correspond to solutions of a differential equation).

Also, systems of linear differential equations very naturally lead to linear transformations where the eigenvectors and eigenvalues play a key role in helping you solve the system, because they "de-couple" the system, by allowing you to think of a complex system in which each of the variables affects the derivative of the others as a system in which you have some new variables that are completely independent of one another (or in the case of generalized eigenvectors, easily dependent on only some of the others). This makes the system easier to solve.

-

This is the best explanation I have seen so far. This short paper not only explains the connection between eigenvalues, eigenvectors and differential equations using very clear, undergraduate math but also has lots of visualizations... and nice examples concerning the development of Romeo's and Juliet's love:

http://wcherry.math.unt.edu/math2700/diffeq.pdf

-