Prove that $\int_0^\pi\frac{\cos x \cos 4x}{(2-\cos x)^2}dx=\frac{\pi}{9} (2160 - 1247\sqrt{3})$ Prove that
$$\int_0^\pi\frac{\cos x \cos 4x}{(2-\cos x)^2}dx=\frac{\pi}{9} (2160 - 1247\sqrt{3})$$
I tried to use Weierstrass substitution but the term $\cos 4x$ made horrible algebraic-forms since $\cos 4x = \sin^4 x + \cos^4 x - 6\sin^2 x \cos^2 x$. My friend suggests me use a contour integration method but I am not familiar with that method. Any idea? Any help would be appreciated. Thanks in advance.
 A: You tried to use Weierstrass substitution but $\cdots$ you have not be patient. Doing it, I arrived to $$\int\frac{\cos x \cos 4x}{(2-\cos x)^2}dx=\int\frac{2 \left(1-t^2\right) \left(t^8-28 t^6+70 t^4-28
   t^2+1\right)}{\left(t^2+1\right)^4 \left(3 t^2+1\right)^2}dt$$ Using partial fraction decomposition, the last integrand is $$-\frac{2882}{3 \left(3 t^2+1\right)}+\frac{776}{3 \left(3
   t^2+1\right)^2}+\frac{320}{t^2+1}+\frac{192}{\left(t^2+1\right)^2}+\frac{64}{\left
   (t^2+1\right)^3}+\frac{128}{\left(t^2+1\right)^4}$$ Some of the integrals are simple. I let you the pleasure of the other.
I must confess that I would prefer something like  contour integration (what I cannot do !!).
A: Proposition : 

\begin{equation}\int_0^\pi\frac{\cos mx}{p-q\cos x}\, dx=\frac{\pi}{\sqrt{p^2-q^2}}\left(\frac{p-\sqrt{p^2-q^2}}{q}\right)^m\qquad\hbox{for}\qquad |q|<p
\end{equation}


Proof :
We have
\begin{equation}
\int_0^\pi\frac{\cos mx}{a^2-2ab\cos x+b^2}\, dx=\frac{\pi}{a^2-b^2}\left(\frac{b}{a}\right)^m\qquad\hbox{for}\qquad |b|<a\tag1
\end{equation}
The complete proof is given by Prof. Omran Kouba and can be seen here.
Now, let $p=a^2+b^2$ and $q=2ab$, then $p+q=\sqrt{p+q}$ and $p-q=\sqrt{p-q}$. Therefore
\begin{align}
2a&=\sqrt{p+q}+\sqrt{p-q}\\[10pt]
2b&=\sqrt{p+q}-\sqrt{p-q}\\[10pt]
a^2-b^2&=\sqrt{p^2-q^2}\tag2\\[10pt]
\frac{b}{a}&=\frac{p-\sqrt{p^2-q^2}}{q}\tag3
\end{align}
then plugging in $(2)$ and $(3)$ to $(1)$ we prove our proposition. $\quad\square$
Set $m=4$ and $p=2$ then differentiate the proposition w.r.t. $q$ and take the limit for $q\to1$, we obtain
\begin{align}
\lim_{q\to1}\int_0^\pi\partial_q\left(\frac{\cos 4x}{2-q\cos x}\right)\, dx&=\lim_{q\to1}\partial_q\left(\frac{\pi}{\sqrt{4-q^2}}\left(\frac{2-\sqrt{4-q^2}}{q}\right)^4\right)\\[10pt]
\int_0^\pi\frac{\cos x \cos 4x}{(2-\cos x)^2}dx&=\frac{\pi}{9} \left(\,2160 - 1247\sqrt{3}\,\right)
\end{align}
The last step is confirmed by Wolfram Alpha.
I think differentiating is easier than using contour integration or partial fraction decomposition. (>‿◠)✌
A: I would like to try to tackle the problem with elementary integration techniques though it is a bit tedious. First of all, using double angle formula, we get
$$\cos x\cos 4x= 8 \cos ^{5} x-8 \cos ^{3} x+\cos x$$ and hence
$$
I=\int_{0}^{\pi} \frac{8 \cos ^{5} x-8 \cos ^{3} x+\cos x}{(2-\cos x)^{2}} d x
$$
By division, we reduce the integrand to a proper fraction,
$$
\begin{aligned}
I \displaystyle &=\int_{0}^{\pi}\left[8 \cos ^{3} x+32 \cos ^{2} x+88 \cos x+224\right] d x + \displaystyle \int_{0}^{\pi} \frac{545 \cos x-896}{(\cos x-2)^2} d x \\&= 240 \pi+545\underbrace{\int_{0}^{\pi} \frac{d x}{\cos x-2}}_{J}+194 \underbrace{\int_{0}^{\pi} \frac{d x}{(\cos x-2)^{2}}}_{K} 
\end{aligned}
$$
For any $a>1,$ let’s define a definite integral$$
\begin{aligned}
\displaystyle I(a):&=\int_{0}^{\pi} \frac{d x}{a-\cos x} \stackrel{x\mapsto \pi-x}{=} \int_{0}^{\pi} \frac{d x}{a+\cos x}\\
\\2 I(a)&=\int_{0}^{\pi}\left(\frac{1}{a-\cos x}+\frac{1}{a+\cos x}\right) d x\\
&=4 a \int_{0}^{\frac{\pi}{2}} \frac{1}{a^{2}-\cos ^{2} x} d x\\
&=4 a \int_{0}^{\frac{\pi}{2}} \frac{\sec ^{2} x}{a^{2} \sec ^{2} x-1} d x\\
&=4 a  \int_{0}^{\frac{\pi}{2}} \frac{d(\tan x)}{a^{2} \tan ^{2} x+\left(a^{2}-1\right)}\\
&=\frac{4}{\sqrt{a^{2}-1}}\left[\tan ^{-1}\left(\frac{a \tan x}{\sqrt{a^{2}-1}}\right)\right]_{0}^{\frac{\pi}{2}}\\
&=\frac{2\pi}{\sqrt{a^{2}-1}}\\
\therefore \quad \int_{0}^{\pi} \frac{d x}{a-\cos x}&=\frac{\pi}{\sqrt{a^{2}-1}} 
\end{aligned}
$$
In particular, $$J=-I(2)=-\frac{\pi}{\sqrt{3}}$$
For $K$, we just differentiate $I(a)$ w.r.t. $a$.
$$
\begin{aligned}
I^{\prime}(a)=-\frac{\pi}{\left(a^{2}-1\right)^{\frac{3}{2}}} 
&\Rightarrow \int_0^{\pi}\frac{1}{(a-\cos x)^{2}} d x=\frac{\pi a}{\left(a^{2}-1\right)^{\frac{3}{2}}} \\
\therefore K=\int_{0}^{\pi} \frac{1}{(2-\cos x)^{2}} d x&=\frac{2 \pi}{3 \sqrt{3}} \\
\end{aligned}
$$
Now we can conclude that
$$I =240 \pi+545\left(-\frac{\pi}{\sqrt{3}}\right)+194\left(\frac{2 \pi}{3 \sqrt{3}}\right) =\frac{\pi}{9}(2160 - 1247 \sqrt{3})$$
A: Contour integration is a bit less painful. For first, it is better to write our integral as:
$$ I = \frac{1}{4}\int_{0}^{2\pi}\frac{\cos(3x)+\cos(5x)}{(2-\cos x)^2}\,dx, $$
then, since $\cos x = \frac{e^{ix}+e^{-ix}}{2}$, by setting $z=e^{ix}$ we get:
$$ I = -\frac{i}{4}\left(\oint\frac{2(z^6+1)}{z^2(z^2-4z+1)^2}\,dz + \oint\frac{2(z^{10}+1)}{z^4(z^2-4z+1)^2}\right)$$
where the path of integration is the unit circle. Since the roots of $z^2-4z+1$ occur in $2\pm\sqrt{3}$, by computing the residues in $z=0$ and $z=2-\sqrt{3}$ (tedious but straightforward) we arrive at the wanted conclusion.
