Question about Method of Undetermined Coefficients? Here's the problem: 
Consider the DE $y''−5y'+6y=e^t\cos(2t)+e^{2t}(3t+4)\sin(t).$
Determine a suitable form $Y (t)$ if the method of undetermined
coefficient is to be used.
So I understand that I'm supposed to come up with a "guess" for $Y(t)$ which contains an undetermined coefficient. For instance, if it were simply $y''−5y'+6y=e^t,$ I would begin with $Y(t)=Ae^t.$ Then I would take the derivative of $Y(t)$ twice and plug that back in to the original DE. 
I also realize that when $\cos/\sin$ are involved, you need to include both (like if it were $y''−5y'+6y=\cos(2t),\; Y(t)$ would be $A\cos(2t)+B\sin(2t)$). But since they're already both in $g(t),$ I don't need to do that, right? 
Right now my initial guess for $Y(t)$ is $Ae^t\cos(2t)+Be^{2t}(3t+4)\sin(t)$... Is this correct? 
 A: I'll highlight the two terms of the function in the right-hand side with coloring:
$$y''−5y'+6y=\color{blue}{e^t\cos2t}+\color{purple}{e^{2t}(3t+4)\sin t} \tag{$*$}$$
Now for a particular solution:

I also realize that when cos/sin are involved, you need to include both (like if it were y''−5y'+6y=cos2t, Y(t) would be Acos2t+Bsin2t). But since they're already both in g(t), I don't need to do that, right? 

You're right that sine and cosine both appear in the right-hand side, but multiplied with different exponential functions! To be on the safe side, it's worth considering the blue and purple terms in $(*)$ separately and carefully following the instructions about what particular solution(s) to suggest. Before you add them, you could check if you have redundant terms.


*

*For the blue part in $(*)$, you suggest a solution of the form:
$$\color{blue}{y_{p_1} = e^t\left( A\cos 2t + B \sin 2t \right)}$$

*For the purple part in $(*)$, the complete form of the standard suggestion would be:
$$\color{purple}{y_{p_2}= e^{2t}\left( Ct\sin t+D\sin t + Et\cos t+F\cos t\right)}$$
Because of linearity or the superposition principle, you can add your suggestions ($y_p = \color{blue}{y_{p_1}}+\color{purple}{y_{p_2}}$) and find the six undetermined coefficients $A,B,C,D,E,F$ by substitution.
A: The solutions for the homogeneous equation are $e^{2t}$ and  $e^{3t}$.
The particular solution takes the form of $$e^{t} [Acos(2t) + B sin(2t)] +te^{2t}[Csin(t) +D cos(t)]$$
We find A, B and C,D by plugging the particular solution in the inhomogeneous equation. 
A: try $$y_P=Ae^{t}\left(B\cos(2t)+C\sin(2t)+((Dt+E)\cos(t)+(Ft+G)\sin(t))e^t\right)$$
A: It's better to substitute $y=e^tz(t)$ and simplify the given equation
$$y'=e^t(z'+z) \implies y''=e^t(z''+2z'+z)$$
The equation becomes simply
$$z''-3z'+2z=\cos(2t)+e^t(3t+4)\sin(t)$$
For the $\cos(2t)$ part, you should choose $z_p=A\cos(2t)+B\sin(2t)$
