Solve $y''+5y'+6y=e^{-3x}$ so I took the auxiliary equation and solved it: $$m^2+5m+6=0$$
gave me
$$ 
m=-2,-3$$
and gives the complementary/general solution
$$c_1e^{-2x}+c_2e^{-3x}$$
so I took
$$axe^{-3x}$$ as the guess
and got the particular solution as
$$−xe^{−3x}$$
and adding that to the general solution gives
$$c_1e^{-2x}+c_2e^{-3x}−xe^{−3x}$$
 A: If you "multiply" the differential equation by the differential operator $(D+3)$, you get the homogeneous equation
$$
(D+3)(D^2+5D+6) y = 0 \Leftrightarrow (D+3)^2 (D+2) y = 0,
$$
which has the general solution
$$
y = A_1 e^{-3x} + A_2 x e^{-3x} + B_1 e^{-2x}
$$
Since $A_1 e^{-3x} + B_1 e^{-2x}$ solves the original homogeneous equation, you must compute $A_2$ in such a way that this solves the original differential equation.
A: Instead $A e^{-3x}$ for a particular, you should use $A(x)e^{-3x}$ so after substituting into the complete ODE we have
$$
A''(x)-A'(x)-1 = 0
$$
with solution
$$
A(x) = -x + c_1 e^x + c_2
$$
but as we are looking for a particular, we can make $c_1=c_2=0$ so finally
$$
A(x) = -x
$$
and a particular solution is $$y_p(x) = -xe^{-3x}$$
A: Variation of parameters method. Somehow, it is hard to memorize the system of equations to solve although it looks very symmetric. You may see them here.
For $a(x)y''+b(x)y'+c(x)y=f(x)$ we assume a particular solution $y_p=u_1y_1+u_2y_2$ where $y_1,y_2$ are homogenous solutions. Then $u_1, u_2$ satisfy the system:
$$y_1u_1'+y_2u_2'=0$$
$$y_1'u_1'+y_2'u_2'=\frac{f(x)}{a(x)}$$
In our example, $y_1=e^{-2x}$, $y_2=e^{-3x}$, $a(x)=1$, $f(x)=e^{-3x}$ and the system to solve:
$$e^{-2x}u_1'+e^{-3x}u_2'=0$$
$$-2e^{-2x}u_1'-3e^{-3x}u_2'=e^{-3x}$$
We find $u_1=-e^{-x}$ and $u_2=-x$. Hence, $y_p=-e^{-3x}-xe^{-3x}$ and we may set $y_p^{new}=-xe^{-3x}$.
A: Exponential functions are eigenfunctions of differential operator. Therefore, one can usually deduce that the particular solution is also the same exponential function but multiplied with a coefficient (value of which needs to be determined, hence the name!). As one of the comment pointed out, it's not as simple as this for this particular case, let's see step by step below
Substitution to Change into First Order Equation
First we make some substitutions. Pay attention to the coefficients, try to factorise the auxiliary equation and compare, do you see anything interesting?
$$
\begin{aligned}
\text{define:   }v=\frac{dy}{dx}+3y\phantom{x}&\implies\phantom{x}\frac{dv}{dx}+2v=e^{-3x}
\end{aligned}
$$
Solve Resulting First Order ODE
Multiply the right equation with integrating factor $e^{2x}$ and integrate. You may want to review the steps to solve first order ODE. Notice that the constant from integration becomes the first homogenous solution?
$$
\frac{d}{dx}\left(e^{2x}v\right)=e^{-x}\phantom{x}\implies\phantom{x}e^{2x}v=-e^{-x}+C\phantom{x}\implies\phantom{x}v=-e^{-3x}+Ce^{-2x}
$$
Solve The Other First Order ODE
Substitute to our definition of $v$, multiply with integrating factor, integrate.
$$
\frac{d}{dx}\left(e^{3x}y\right)=-1+Ce^{x}\phantom{.}\implies\phantom{.}e^{3x}y=-x+C_{1}e^{x}+C_{2}\phantom{.}\implies\phantom{.}y=-xe^{-3x}+C_{1}e^{-2x}+C_{2}e^{-3x}
$$
Final Remark
Hope you can see why, when $e^{-3x}$ is also in the homogenous solution we then use $Axe^{-3x}$ as our particular solution instead of $Ae^{-3x}$. I hope you can also generalise the case for higher order ODEs
