# Eigen vectors and their relationship to (first order) linear differential equations

Ok, so in my DiffEq class we've been doing problems which more or less amount to solving equations of the form:

$$\frac{dY}{dt} = AY$$

Where $A$ is just some $2\times2$ linear transformation and $Y$ is a parametric vector function defined more specifically as

$$Y(t) = \begin{bmatrix} x(t) \\ y(t) \end{bmatrix}$$

The end result, assuming that there exists $\lambda_1, \lambda_2 \ne 0; \lambda_1 \ne \lambda_2$ which define the eigen values for A, is a definition for $Y(t)$ of the form,

$$Y(t) = k_1e^{\lambda_1t}\vec{V_1} + k_2e^{\lambda_2t}\vec{V_2}$$

Where $\vec{V_1}, \vec{V_2}$ are the corresponding eigen vectors to their respective eigen values and $k_1, k_2$ are just some constants.

For solutions which involve either $k_1 = 0$ or $k_2 = 0$, the end result is a straight-line solution. The rest are exponential curves within the vector space defined by the eigen vectors.

My understanding of eigen vectors, from a linear algebra class I took a year ago, so far is as follows (roughly):

• geometrically speaking, an eigen vector is any vector whose direction after transformation by some matrix $A$ remains the same. It's only scaled and/or negated.

• Every eigen vector for some matrix $A$ composes a subspace which in turn defines the eigen space for $A$'s vector basis.

• therefore, the eigen vectors which $span(A)$ are linearly independent and define a coordinate space which also exists within $A$.

Regardless of whether or not the above is correct (if there's a mistake, any clarification/correction would be appreciated), what is it about eigen vectors specifically which allows for them to be used to solve these forms of differential equations?

Consider the vector ODE $$\frac{d Y}{dt} = AY,\;\; Y(0)=Y_0,$$ where $Y$ is an $N$-vector function of $y$, where $Y_0$ is a constant $N$-vector, and where $A$ is a constant $N\times N$ matrix. This has a unique solution $$Y(t) = e^{tA}Y_0\; \mbox{where}\; e^{tA}=\sum_{n=0}^{\infty}\frac{1}{n!}t^nA^n.$$ If $Y_0$ is an eigenvector of $A$ with eigenvalue $\lambda$, then $$e^{tA}Y_0 = \sum_{n=0}^{\infty}\frac{1}{n!}t^nA^nY_0 = \left(\sum_{n=0}^{\infty}\frac{1}{n!}\lambda^n t^n\right)Y_0 = e^{\lambda t}Y_0.$$ If $A$ has a basis of eigenvectors $Y_1$, $Y_2$, then all solutions can be formed in this way because $Y_0 = \alpha Y_1+\alpha_2Y_2$ for unique $\alpha_1$, $\alpha_2$, which leads to the solution of the ODE in the form $$Y(t)=\alpha_1 e^{\lambda_1 t}Y_1+\alpha_2 e^{\lambda_2 t}Y_2.$$ That's the basic story for this problem.

The History of eigenvector analysis traces back to Fourier's analysis of his Heat Equation, where he used separation of variables to solve $$\frac{\partial u}{\partial t} = \frac{\partial^2u}{\partial x^2}.$$ His separation of variables technique involved finding all solutions of the form $u(t,x,y,z)=T(t)X(x)$, and trying to form a general solution out of sums of such solutions. That naturally led to eigenfunction/eigenvalue equations, long before the terminology even existed. Fourier looked for solutions of the form $u(t,x,y,z)=T(t)X(x)$ and he rearranged the terms to conclude that $$\frac{T'(t)}{T(t)} = \frac{X''(x)}{X(x)}$$ In order for there to exist such a solution, he deduced that, for fixed $t$, the right side would have to be constant, which would then force the left side to be a constant as well. So he introduce a separation parameter $\lambda$ which became an eigenvalue of $\frac{d}{dt}$, and of $\frac{d^2}{dx^2}$: $$\frac{T'(t)}{T(t)}=\lambda,\;\;\; \lambda=\frac{X''(x)}{X(x)}.$$ Fourier then went on to try to find the most general solution by superimposing linear combinations of the possible eigenvalue solutions: \begin{align} u(t,x) &= e^{\lambda_1 t}(A_1\cos(\sqrt{\lambda_1}x)+B_1\sin(\sqrt{\lambda_1}x)) \\ &+e^{\lambda_2 t}(A_2\cos(\sqrt{\lambda_2}x)+B_2\sin(\sqrt{\lambda_2}x))+\cdots. \end{align}

Fourier's superposition was the precursor of the first general definition of linearity and a linear space, and it was first done here in the context of infinite-dimensional linear spaces of functions.

Fourier conjectured that solutions of such equations could be written in terms of such separated solutions, which is now translated into finding a basis of eigenvectors. The development of linearity, linear operators, selfadjoint operators, orthogonal eigenvector expansions, and eigenvector analysis evolved directly from the work of Fourier. And the infinite-dimensional problems of Fourier came before the finite-dimensional cases, making it even more confusing and obscure because its Historical context is lost when studying such analysis for the first time in finite-dimensional Linear Algebra.

Observe that $A = S \Lambda S^{-1}$, where $S = \begin{bmatrix}V_1 & V_2\end{bmatrix}$, $\Lambda = \begin{bmatrix}\lambda_1 & 0 \\ 0 & \lambda_2\end{bmatrix}$, and $S^{-1}=S^T$. Consequently, $$\frac{dY(t)}{dt}=AY(t) = S\Lambda S^{-1} Y(t). \tag{1}$$ Let $S^{-1}Y(t)=Z(t)$. Consequently, $(1)$ becomes $$\frac{dZ(t)}{dt}=\Lambda Z(t) \implies \begin{bmatrix}z_1'(t) \\ z_2'(t)\end{bmatrix}=\begin{bmatrix}\lambda_1 z_1(t)\\\lambda_2 z_2(t)\end{bmatrix} \implies Z(t)=\begin{bmatrix}k_1e^{\lambda_1t}\\k_2e^{\lambda_2t}\end{bmatrix}. \tag{2}$$ But $Y(t)=S Z(t)=\begin{bmatrix}V_1 & V_2\end{bmatrix}\begin{bmatrix}k_1e^{\lambda_1t}\\k_2e^{\lambda_2t}\end{bmatrix}=?$

$Y' = A Y\\ Y = e^{At}Y_0$

$e^{At} = \sum_\limits{n=0}^\infty \frac {A^nt^n}{n!}$

$A = PDP\\ A^n = PD^nP^{-1}$

$e^{At} = P\left(\sum_\limits{n=0}^\infty \frac {D^nt^n}{n!}\right)P^{-1}\\ Y = Pe^{Dt}P^{-1}Y_0$

$e^{Dt} = \begin{bmatrix} e^{\lambda_1 t}\\&e^{\lambda_2 t}\end{bmatrix}$

$PY_0 = \begin {bmatrix} k_1\\k_2\end{bmatrix}$

$Y = k_1V_1 e^{\lambda_1 t} + k_2V_2 e^{\lambda_2 t}$

The basic idea here is to find a new set of variables, $X(t)$ and $Y(t)$, related to the original variables $x(t)$ and $y(t)$ by a linear transformation, so that the differential equations for $X(t)$ and $Y(t)$ are decoupled: $$\dot X = \lambda_1 X\, ,\qquad \dot Y=\lambda_2 Y\, . \tag{1}$$ In other words, assuming $$\left(\begin{array}{c} x\\ y \end{array}\right)= M \left(\begin{array}{c} X\\ Y\end{array}\right)$$ where $M$ is invertible, we have \begin{align} \frac{d}{dt}\left(\begin{array}{c} x\\y \end{array}\right)&= A \left(\begin{array}{c} x \\ y\end{array}\right)\, ,\\ M \frac{d}{dt} \left(\begin{array}{c} X \\ Y\end{array}\right)&= A M \left(\begin{array}{c} X\\Y\end{array}\right)\, ,\\ \frac{d}{dt}\left(\begin{array}{c} X\\ Y\end{array}\right)&= M^{-1} A M \left(\begin{array}{c} X\\ Y\end{array}\right)\, .\tag{2} \end{align} Thus if $M^{-1} A M$ is diagonal with non-zero entries $\lambda_1,\lambda_2$, the system (2) is solved by (1), and $\lambda_1,\lambda_2$ are the eigenvalues of $A$ (by construction). The new coordinates $X$ and $Y$ are by construction eigenvectors of $A$. These new coordinates are “special” in that they have an especially simple evolution. Having solved in terms of the special coordinates $X,Y$, one can then go back to the original variable $x,y$ using $M$.