Particular solutions for a linear 2nd order ODE Say we had $$y'' +2y' -8y = xe^{-x}.$$ For the particular solution, would it be wise to try $$y_p(x) = (c_1 + c_2x)e^{-x}\quad or\quad y_p(x) = c_1xe^{-x}?$$ Similarly, we had $y'' +2y' -8y = x^2e^{-x}$ or $ y'' +2y' -8y = x^2cos(x)$ would it be wise to try $y_p(x) = (c_1 + c_2x + c_3x^2)e^{-x} $ and $y_p(x) = (c_1 + c_2x + c_2x^2)(\sin(x) + \cos(x))$?
 A: A good criteria:

Let $a_ny^{(n)}+a_{n-1}y^{(n-1)}+\cdots+a_1y'+a_0y=Q(x)$ where $a_n\neq0$ and $Q(x)$ is not zero in an interval $I$.  If No term of $Q(x)$ is the same as a term of $y_c(x)$, then the particular solution $y_p(x)$ is a linear combination of the terms in $Q(x)$ and all its linearly independent derivatives.

Here, we have $y_c(x)=C_1e^{2x}+C_2e^{-4x}$ so according to what above point says we get: $$y_p(x)=(Ae^x+Bxe^x)$$ Now we can use the Undetermined coefficient method to find $A$ and $B$.
A: In general, if you have an equation of the form
$$
y^{(n)}+a_{n-1}y^{(n-1)}+\cdots a_0 y^{(0)}=p(x)\,\mathrm{e}^{ax},
$$
where $p$ is a polynomial of degreee $k$, then you look for a solution of the form $q(x)\,\mathrm{e}^{ax}$, where $q$ is also a polynomial of degree $k$, unless $a$ is a root of the characteristic polynomial. In the latter case you look for a solution of the form $q(x)\,\mathrm{e}^{ax}$, where $q$ is a polynomial of degree $k+m$, where $m$ is the multiplicity of the root $a$ in the characteristic polynomial.
In your case, you look for a special solution of the form $y=(A+Bx)\mathrm{e}^{-x}$, but the general solution is of the form:
$$
y=(A+Bx)\mathrm{e}^{-x}+c_1\mathrm{e}^{2x}+c_1\mathrm{e}^{-4x}
$$
For $x^2\cos x$ you should look for a special solution of the form
$$
p_1(x)\cos x+p_2(x)\sin x,
$$
where $p_1$, $p_2$ are polynomials of degree at most 2.
A: For the first RHS the particular solution is of the form $(c_1+c_2x)e^{-x}$.
For the second RHS the appropriate particular solution is $(c_1+c_2x+c_3x^2)e^{-x}$.
For the third RHS the appropriate particular solution is $(c_1+c_2x+c_3x^2)\sin(x)+(d_1+d_2x+d_3x^2)\cos(x)$.
