Annihilator method, $y''-4y'-5y=x^2 e^{-x} +6e^{-x}$ The equation $y''-4y'-5y=x^2 e^{-x} +6e^{-x}$, has a characteristic polynomial  $$(r^2 -4r -5)$$
with roots $-1$, $5$. Our operator $M(y)$, which is the equation to which the RHS is a solution of, has a characteristic polynomial $$p(r)=(r+1)^3  (r+1)=(r+1)^4$$
which has roots $-1$ with multiplicity of four. The relevant polynomial for this problem is then $$(r+1)^4 (r^2 -4r-5)$$
which has roots $-1,-1,-1,-1,-1,5$. And a particular solution of the differential equation is then $$\psi_p=c_1e^{-x}+c_2xe^{-x}+c_3x^2e^{-x}+c_4x^3e^{-x}$$
, where $c_i$ are constants to be determined. Notice that we can remove two terms because they are solutions of $L(y)=0$. Is this correct? I'm not sure how to deal with repeated roots.
 A: $$y''-4y'-5y=x^2 e^{-x} +6e^{-x}$$
The homogeneous solution of the DE is:
$$\implies y_h=c_1e^{-x}+c_2e^{5x}$$
And the guess for the particular solution should be:
$$y_p=(ax^2+bx+c)e^{-x}\color {red}{x}$$
$$y_p=(ax^3+bx^2+cx)e^{-x}$$

Annihilator method:
$$y''-4y'-5y=x^2 e^{-x} +6e^{-x}$$
$$ (D^2 -4D-5)y=x^2 e^{-x} +6e^{-x}$$
$$ (D^2 -4D-5)y=(x^2  +6)e^{-x}$$
$$\color{red}{(D+1)^3} (D^2 -4D-5)y=0$$
$$\color{red}{(D+1)^3} (D+1)(D-5)y=0$$
With the annihilator method you should get for the particular solution:
$$y_p=c_1xe^{-x}+c_2x^2e^{-x}+c_3x^3e^{-x}$$
$$y_p=e^{-x}(c_1x+c_2x^2+c_3x^3)$$
And for the homogeneous solution:
$$y_h=Ae^{-x}+Be^{5x}$$
So your solution  looks good :
$$\psi_p=c_1e^{-x}+c_2xe^{-x}+c_3x^2e^{-x}+c_4x^3e^{-x}$$
But the first term is absorbed by the homogeneous solution so that you only have three constant.
$$\psi_p=c_2xe^{-x}+c_3x^2e^{-x}+c_4x^3e^{-x}$$
A: Make your life easier from the starting point using $y=z\, e^x$. This gives
$$z''-6 z'=x^2+6$$ Reduction of order makes the problem quite simple.
