Linear Stability Analysis of ODEs/PDEs I'm looking for a systematic understanding/approach to linear stability analysis of differential equations. I'm interested in an arbitrary (non-linear) PDE, $\mathcal L u=0$ (or system of PDEs although probably best to start with just one, and most conclusions generalise straightforwardly). Obviously the domain and BCs are important, which I'll mention in a moment.
Our procedure is standard: given a steady-state solution, $u_0$, we perturb $u=u_0+\epsilon v$ where $\epsilon\ll1$, and consider $\mathcal L (u_0+\epsilon v)-\mathcal L u_0=0$ and we ignore terms of $O(\epsilon^2)$. For concreteness let's say we're working in 3D space and time.
Basically my question is what form to assume for $v$. Sometimes I see the ansatz $v(x,y,z,t)=\phi(y)e^{kx+\ell y + \omega t}$. Sometimes $v=e^{kx+\ell y +m z + \omega t}$. Sometimes the time component (or spatial components) has an imaginary part, e.g. $e^{i\omega t}$. Obviously multiplying by a constant doesn't matter, but this seems to be more common when $\mathcal L$ is second-order in $\partial/\partial t$, and I can't quite see why that's more convenient. Also, if the frequencies/wavenumbers are real, then multiplication by a constant does matter.
I understand each stability calculation that I encounter, but when I come to perform my own, I have less rigorous an idea of the ramifications of various choices. It is clear that an exponential ansatz is related to taking a Fourier transform (equivalent? Even when the coefficients of the differential operators in $\mathcal L$ depend on the independent variables?), so one can only do so for independent variables whose domain is infinite. Were the domain finite and periodic one could presumably constrain the constants to be integers. I'm not sure what to do in other situations. This also assumes the solution is separable, and that all relevant perturbations are in the span of this Fourier basis set, which might be implicitly assuming something about the eigenspace of $\mathcal L$ and the BCs. The ansatz also seems to implicitly assume something about the initial condition of the perturbation. There seem to be a lot of nuances here that I am not crystal clear on.
Is there a collection of examples, or a reference book that lays any of this out clearly/systematically, with a discussion on caveats, assumptions, pitfalls?
 A: This is a non-rigorous answer that gives the rough idea.
What you are doing is linearizing the operator $\mathcal L$:
$$ \mathcal L(u_0+\epsilon v) = \mathcal L(u_0) + \epsilon \mathcal M(v) + O(\epsilon^2) ,$$
where $M$ is a linear operator.  (You can think of it as the derivative of $\mathcal L$ at $u_0$.)
Then it is generally true that the stability of $\mathcal L$ is more or less equivalent to the stability of $ \mathcal M $.  And the stability of $\mathcal M$ depends upon its spectrum, that is, (more or less) its eigenvalues.
Thus we are searching for functions $v \ne 0$ such that $ \mathcal M v = \lambda v$, and if all such solutions satisfy $\lambda < 0$, this (usually) means stability, and if there exists any solution with $\lambda >0$, this (usually) means instability.  This follows by considering the differential equation
$$\frac{\partial u}{\partial t} = M(u), $$
whose solutions will generally be $u(t) = e^{ \lambda t} v$.
If $\mathcal L$ is a differential operator whose coefficients are constant, then $\mathcal M$ will be a linear differential operator whose coefficients are constants.  In that case, generally the eigenvectors $v$ will be exponentials, for example
$$ v( x, y) = e^{k x + l y} .$$
If you know (from the boundary conditions) that the solutions are bounded, this will restrict $k$ and/or $l$ to be purely imaginary.  (If the domain isn't all of $\mathbb R$, then it might be linear cominations of exponentials, like hyperbolic functions or trig functions.)
If, say, $\mathcal L$ is a differential operator in the variables $x$ and $y$, and the coefficients depend only upon $y$, then the general form of the eigenvectors will be
$$ v(x,y) = \phi(y) e^{k x} .$$
If $\mathcal L$ is an operator on a finite dimensional space (so this excludes differential equations), then this stuff can generally be easily be made rigorous.  In that case $\mathcal M$ is the matrix of partial derivatives of $\mathcal L$, also called the Jacobian of $\mathcal L$.  On infinite dimensional spaces, if you want to see all this done rigorously, then you need to learn functional analysis.
