Why is the general solution to a linear differential equation sought in the form of $Ce^{kx}$? Why is the solution to linear differential equations with constant coefficients sought in the form of $Ce^{kx}$ ?
 A: I'm making a long reply to one of joo's comments:
Well, a linear differential equation with constant coefficients has the form $y' = Ay$ for some matrix $A$. Now the general solution with the initial value $y(0) = v$ is $y(t) = \exp(tA)v$, where $\exp(X) = \sum_{k=0}^\infty \frac 1{k!} X^k$ for all square matrices $X$ (it can be proved that this series is always convergent, just like the ordinary exponential series). This follows from the formula
$$
\tfrac d{dt}\exp(tA) = A\exp(tA),
$$
which is not at all trivial, but which is proved in much literature on the subject.
In the case where $A$ is diagonalizable, there exists a basis $v_1,\ldots,v_n$ for $\Bbb R^n$ of eigenvectors for $A$. Putting $V = (v_1,\ldots,v_n)$ in column form, $D:= V A V^{-1} = \begin{pmatrix}\lambda_1 & &0\\ & \ddots\\ 0& & \lambda_n\end{pmatrix}$ is a diagonal matrix, where $\lambda_i$ is the eigenvalue corresponding to $v_i$ (Exercise!). Now
$$
\exp(D)=\exp(V^{-1} A V) = \sum_{k=0}^\infty (V^{-1} A V)^k = \sum_{k=0}^\infty \underbrace{(V^{-1}AV)(V^{-1}AV)\cdots (V^{-1} A V)}_{\text{$k$ times}} = V^{-1}\big(\sum_{k=0}^\infty A^k\big)V = V^{-1}\exp(A)V.
$$
Similarly, $\exp(tD)=\exp(t V^{-1}AV)=V^{-1}\exp(tA)V$ for all $t\in\Bbb R$. Hence if $y$ is the solution from before, we have
$$
y = \exp(tA)v=V\exp(tD)V^{-1}v = V\begin{pmatrix}e^{\lambda_1 t} & & 0\\&\ddots&\\0&&e^{\lambda_n t}\end{pmatrix}V^{-1} v = \big(\sum_{i=1}^n e^{\lambda_i t}v_i\big) V^{-1} v,
$$
where the above expression for $\exp(tD)$ follows from the fact that $D^k = \begin{pmatrix}\lambda_1^k & &0\\ & \ddots\\ 0& & \lambda_n^k\end{pmatrix}$ for all $k\in\Bbb N$; hence $\exp(tD)$ is calculated entry-wise. Now the expression on the right is a linear combination of functions of the form $t\mapsto e^{\lambda_i t} v_i$, which was what you wanted to know.
A: Short answer: because it works, i.e., solutions of this form do exist (if we allow complex $k$). (Though they do not always form a basis of solutions, which is why we sometimes have to consider $x^j e^{kx}$ as well). 
Perhaps you wanted to know how people came up with the idea of using exponential functions for this purpose. But what is the fundamental idea of exponential growth? It's a function whose rate of change is proportional to the value of the function. In symbols this becomes $y'=ky$, which is a differential equation with constant coefficients.  We can even define exponential functions as solutions of this equation (which is sometimes done). Whether or not this is done, the relation between $e^{kx}$ and $y'=ky$ is the major reason why we care about exponential functions at all. (We could just stick to rational exponents $x^{p/q}$ for all algebra purposes, why have real exponents?)
After seeing the relation of $\exp(kx)$ to first-order equations, it's relatively easy to see that it is related to higher-order equations too: taking a derivative several times just brings out the factor of $k$ several times. 
