Bogoliubov Transform For the operator defined as polynomial is the boson creation and annihilation operators $\hat{a}$, $\hat{a}^\dagger$ such that $[\hat{a},\hat{a}^\dagger] = 1$
$$\hat{L} = A\hat{a}^2 + B\hat{a}^{\dagger2} + C\hat{a}^\dagger\hat{a} + D\hat{a}^\dagger + E\hat{a} + F,$$
How can I introduce the Bogoliubov transformation $\hat{b} = \mu\hat{a} +\nu\hat{a}^\dagger$, such that the operator $\hat{L}$ can be written diagonal in $\hat{b}$?
 A: It is not quite clear to me what diagonal means for you, I'll assume you mean this
$$ \hat{L} = G \hat{b}^\dagger \hat{b} + H\hat{b}+ J\hat{b}^\dagger + K. $$
Now we can just expand and compare coefficients. We have
$$ \hat{b}^\dagger \hat{b} 
= (\mu \hat{a} + \nu \hat{a}^\dagger) (\mu \hat{a} + \nu \hat{a}^\dagger)
= \mu^2 \hat{a}^2 + \nu^2 {a}^{\dagger 2} + \mu \nu \hat{a}^\dagger \hat{a} + \mu \nu \hat{a} \hat{a}^\dagger. $$
Now we use the anticommutation rule and get
$$ \hat{b}^\dagger \hat{b} 
= \mu^2 \hat{a}^2 + \nu^2 {a}^{\dagger 2} + 2\mu \nu \hat{a}^\dagger \hat{a} + \mu \nu. $$
Hence, we have
$$ G \hat{b}^\dagger \hat{b} + H\hat{b} + J\hat{b}^\dagger + K
= G\mu^2 \hat{a}^2 + G\nu^2 {a}^{\dagger 2} + 2G\mu \nu \hat{a}^\dagger \hat{a} + G\mu \nu + H\mu \hat{a} + H\nu \hat{a}^\dagger + J \nu \hat{a} + J\mu \hat{a}^\dagger + K.  $$
Collecting terms gives
$$  G \hat{b}^\dagger \hat{b} + H\hat{b} + J\hat{b}^\dagger + K
= G \mu^2 \hat{a}^2 + G \nu^2 \hat{a}^\dagger + 2 G \nu \mu \hat{a}^\dagger \hat{a} + (H\mu + J \nu) \hat{a} + (H\nu + J \mu) \hat{a}^\dagger+ (G\mu \nu + K).  $$
If we want
$$  G \hat{b}^\dagger \hat{b} + H\hat{b} + J\hat{b}^\dagger + K 
= \hat{L} 
= A \hat{a}^2 + B \hat{a}^{\dagger 2} + C \hat{a}^\dagger \hat{a} + D \hat{a}^\dagger + E\hat{a} + F, $$
then we have to match the coefficients, i.e.
$$ \begin{cases}
A &= G \mu^2,\\
B &= G \nu^2, \\
C &= 2G\mu \nu,\\
D &= H\mu + J \nu,\\
E &= H\nu + J \mu,\\
F &=  G\nu \mu + K.
\end{cases} $$
This gives you also some restrictions on when you can write it in diagonal form.
