When solving a differential equation, why do we always start with guessing the solution is exponential function? For example, when solving second order differential equation with constant coefficients, we start with guessing the solution is the form of the linear combination of two independent exponential functions. I read some explanations saying it's for the convenience of integral and derivation.  Is there any more reason? 
I'm still an undergraduate student, please write an answer in a way that I can understand.
 A: Imagine I want to solve the first order ODE $y' = ky$. By seperation of variables, the solution is easily deduced to be $y = Ce^{kt}$.
For reasons that will become clear later, I will introduce the notation $Dy = y'$. The symbol $D$ can be thought of as a differential operator. Thus, saying $y_0 = Ce^{kt}$ is a solution to $y' = ky$ is the same as saying $y_0' = Dy_0 = ky_0$ so $(D-k)y_0 = 0$. (In the language of linear algebra, the function $e^{kt}$ is said to be an eigenfunction of the differential operator $D$.)
Now suppose I want to solve $y'' - 3y' + 2y = 0$. This can be written as $(D^2 - 3D + 2)y = 0$. Being ambitious, we might wonder what would happen if we factored the polynomial of differential operators $D^2 - 3D + 2$ as $(D-2)(D-1)$.
But as we showed above we have $(D-1)e^{t} = 0$ and $(D-2)e^{2t} = 0$ so the polynomial $(D-2)(D-1)$ annihilates all exponentials of the form $e^t$ and $e^{2t}$. Further, since $(D-2)(D-1)$ is a "linear operator", we have that any linear combination $C_1 e^t + C_2 e^{2t}$ is also annihilated by $(D-2)(D-1)$
$$
(D-2)(D-1) [C_1 e^t + C_2 e^{2t}] = C_1 (D-2)\underbrace{(D-1)e^t}_{=0} + C_2 (D-1)\underbrace{(D-2)e^{2t}}_{=0} = 0
$$
So why do exponentials appear as basis solutions to homogenous linear ODEs with constant coefficients? Because exponentials are the solution to first order homogeneous linear ODEs and higher order differential operators can be "factored" into first order differential operators.

This is not, by any means, a rigorous proof, but should give you a good idea for why we might expect exponentials to solve ODEs rather than just a guess and check.
A: Let me explain using the second order ODE with constant coefficients. I think that would give you some idea. Consider the ODE $$a_{0}y''(x)+ a_1 y'(x) + a_2 y(x) = 0$$ where $a_i$s are constants and y is a function of x.
The very reason we solve this equation is to find out the function $y(x)$. For such a function y to exist, it must be such that the the sum of $a_0y$, $a_1y'$ and $a_2y''$ must go to zero.
Some of the common functions that we know are polynomial functions, trigonometric functions, logarithmic function and exponential function. Out of all these the property that we want is exhibited by exponential function. 
Let us see how.
Consider $\exp x$. Its derivative is also $\exp x$. With $a_0$ being 1 and $a_1$ being -1, we see that $\exp x - \exp x = 0$, satisfying the ODE: $y'-y=0$. Another instance, consider $\exp (-3x)$. its first derivative is $-3\exp (-3x)$ and second is $9\exp (-3x)$. Then taking $\phi(x) = \exp (-3x)$, we see that $\phi(x)$ is a solution of  $y'' +y' -6y = 0$.
This is how exponential functions become the best choice for the solution of such an equation.
P.S. I have not included any reasoning that involves Linear Algebra topics. You can check for linear independence, especially for functions of type $\exp (ax)$.
