Solving the differential equation $y'' + y = 8\cos(x) \cos(2x)$ I'm solving the differential equation $y'' + y = 8\cos(x) \cos(2x)$
I started to find the homogeneous solution:
We search the zeros of the associated polynomial:
$$r^2 + 1 = 0$$
This yields $r = i$ or $r = -i$
Hence, the homogeneous solution is:
$$y_h = e^0(c_1\cos(x) + c_2\sin(x)) + e^0(c_3\cos(-x) + c_4\sin(-x))$$
And by rewriting this, we find:
$$y_h = c_1\cos(x) + c_2\sin(x)$$
I have trouble making a suggestion for the particular solution. I would suggest something like:
$$y_p = (A\cos(x) + B\sin(x))(C\cos(2x) + D\sin(2x))$$
but the answer my book gives is :
$$y = x_1\cos(x) + (c_2 + 2x)\sin(x) - 1/2\cos(3x)$$
Where does the $\cos(3x)$ come from? This makes me wonder my suggestion won't work. 
Any help will be appreciated.
 A: This is a second order linear nonhomogeneous differential equation:
$$y''+p(x)y'+q(x)y=r(x)$$
You correctly found the complementary solution. Now, to avoid guessing the form of the particular solution using the method of undetermined coefficents, I suggest using Variation of Parameters.

The basis solutions are $\cos{x}$ and $\sin{x}$. Therefore, the Wronskian simply turns out to be:
$$W(x)=\begin{vmatrix} \cos{x} & \sin{x} \\ \frac{d}{dx}(\cos{x}) & \frac{d}{dx}(\sin{x}) \end{vmatrix}=\begin{vmatrix} \cos{x} & \sin{x} \\ -\sin{x} & \cos{x} \end{vmatrix}=1$$
The particular solution will be given by:
$$y_p(x)=v_1(x)\cos{x}+v_2(x)\sin{x} \tag{1}$$
Where:
$$v_1(x)=-\int \frac{r(x) \sin{x}}{W(x)}~dx=-8\int \cos{x}\cdot \cos(2x)\cdot \sin{x}~dx$$
And:
$$v_2(x)=\int \frac{r(x) \cos(x)}{W(x)}~dx=8\int \cos^2{x}\cdot \cos(2x)~dx$$
All that remains to do is to evaluate these integrals, and substitute them into $(1)$. Then, use the fact that $y(x)=y_h(x)+y_p(x)$ to obtain your general solution.
A: I see that you were trying to approach this using the method of undetermined coefficients, therefore I decided to write an answer on this too:

Using one of the Prosthaphaeresis formulas:
$$\cos{a}\cos{b}\equiv \frac{\cos(a-b)+\cos(a+b)}{2}$$
Letting $a=x$ and $b=2x$, we have:
$$\begin{align}\cos(x)\cos(2x)&\equiv \frac{\cos(x-2x)+\cos(x+2x)}{2} \\ &\equiv \frac{\cos(-x)+\cos(3x)}{2} \\ &\equiv \frac{\cos{x}+\cos(3x)}{2} \end{align}$$
Hence, our differential equation can be written as:
$$y''+y=4\cos{x}+4\cos(3x) \tag{1}$$

This is of help, because the particular solution will be the sum of the particular solution to $y''+y=4\cos{x}$, and $y''+y=4\cos(3x)$. For the first one, use the ansatz:
$$y_{p_1}(x)=x(A\cos{x}+B\sin{x})$$
Note that it was multiplied by $x$ because $\cos{x}$ and $\sin{x}$ are already in the complementary solution. For the second one, use the ansatz:
$$y_{p_2}(x)=C\cos(3x)+D\sin(3x)$$
Therefore giving:
$$y_p(x)=y_{p_1}(x)+y_{p_2}(x)=Ax\cos{x}+Bx\sin{x}+C\cos(3x)+D\sin(3x) \tag{2}$$
You can evaluate $A,B,C$ and $D$ by substituting $y_p(x)$ into the differential equation and then equating coefficients.
A: The C.F.$=c_1\cos x+c_2\sin x$,
By inverse operator rule: When
\begin{align*}
P.I.&=\dfrac{1}{f(D^2)}\cos(ax+b)\  \text{ put }D^2=-a^2,\text{ when }[f(-a^2)\neq0]\\
&=x\dfrac{1}{f'(D^2)}\cos(ax+b)\  \text{ put }D^2=-a^2,\text{ when }[f(-a^2)=0,\ f'(-a^2)\neq0]\\
&=x^2\dfrac{1}{f''(D^2)}\cos(ax+b)\  \text{ put }D^2=-a^2,\text{ when }[f'(-a^2)=0,\ f''(-a^2)\neq0]\\
&=\cdots
\end{align*}
So the P.I. is given by (using inverse operator rules),
\begin{align*}
P.I.&=\dfrac{4}{D^2+1}(\cos3x+\cos x)\\
&=4\left[\dfrac{1}{D^2+1}\cos3x+\dfrac{1}{D^2+1}\cos x\right]\\
&=4\left[\dfrac{1}{-3^2+1}\cos3x+x\cdot\dfrac{1}{2D}\cos x\right]\\
&=4\left[-\dfrac{1}{8}\cos3x+\dfrac{x}{2}\cdot\dfrac{1}{D}\cos x\right]\\
&=4\left[-\dfrac{1}{8}\cos3x+\dfrac{x}{2}\sin x\right]\\
&=2x\sin x-\dfrac{1}{2}\cos3x.
\end{align*}
So, $y=C.F.+P.I.=c_1\cos x+c_2\sin x+2x\sin x-\dfrac{1}{2}\cos3x$
or, $y=c_1\cos x+(c_2+2x)\sin x-\dfrac{1}{2}\cos3x$.
