How to find $\int_0^\pi (\log(1 - 2a \cos(x) + a^2))^2 \mathrm{d}x, \quad a>1$? Integration by parts is of no success. What else to try?
$$\int_0^\pi (\log(1 - 2a \cos(x) + a^2))^2 \mathrm{d}x, \quad a>1$$
 A: Let your integral be $L(a)$.  As $a \to +\infty$ we have the asymptotic series
$$ L(a) \sim 4 \pi \ln(a)^2 + \frac{2\pi}{a^2} + \frac{\pi}{2 a^4} + \frac{2\pi}{9 a^6} + \ldots$$
It looks like the coefficient of $a^{-2k}$ is $2 \pi/k^2$.  So it looks like
$$  L(a) = 4 \pi \ln(a)^2 + \sum_{k=1}^\infty \frac{2\pi}{k^2 a^{2k}} = 4\, \left( \ln  \left( a \right)  \right) ^{2}\pi +2\,\pi \,{\it 
polylog} \left( 2,{a}^{-2} \right)$$
Numerically this seems to work: e.g. for $a=2$ Maple (with Digits=20) gets 
$7.7192617649944513786$ for $L(2)$ and $7.7192617649944513785$ for $4\, \left( \ln  \left( 2 \right)  \right) ^{2}\pi +2\,\pi \,{\it 
polylog} \left( 2,1/4 \right)$
A: Factor $a^2$ out in your integral $I(a)$:
$$
I(a)=\int_0^\pi(\log (a^2(1-2a^{-1}\cos\theta+a^{-2}))^2d\theta
$$
$$
=\int_0^\pi(2\log a+\log(1-2a^{-1}\cos\theta+a^{-2}))^2d\theta
$$
$$
=4\pi\log^2a+4\log a\int_0^\pi \log(1-2a^{-1}\cos\theta+a^{-2})d\theta+\int_0^\pi \log^2(1-2a^{-1}\cos\theta+a^{-2})d\theta.
$$
Now observe that the change of variable $u=2\pi-\theta$ yields the same integrals with bounds $\pi$ and $2\pi$ instead, so
$$
I(a)=4\pi\log^2a
$$
$$+2\log a\int_0^{2\pi} \log(1-2a^{-1}\cos\theta+a^{-2})d\theta+\frac{1}{2}\int_0^{2\pi} \log^2(1-2a^{-1}\cos\theta+a^{-2})d\theta.
$$
Now, see this thread. Denoting $\gamma$ the circle of radius $a^{-1}$ and center $0$ in the complex plane, we have
$$
\int_0^{2\pi} \log(1-2a^{-1}\cos\theta+a^{-2})d\theta=\mbox{Re}\frac{2}{i}\int_\gamma\frac{\log(1-z)}{z}dz=0
$$
by Cauchy's integral formula. For the other integral, we get 
$$
\int_0^{2\pi} \log^2(1-2a^{-1}\cos\theta+a^{-2})d\theta=\frac{4}{i}\int_\gamma\frac{\log^2|1-z|}{z}dz.
$$
Now it remains to make the polylog/dilogarithm invoked by Robert Israel and Sasha appear, or to wait for someone to find the clever elementary argument which would be more appropriate for a contest-math question. We obtain:

$$
I(a)=4\pi\log^2a+2\pi\mbox{Li}_2(a^{-2}).
$$

A: A complex idea:
$$z=ae^{ix}\in\Bbb C\;,\;a\in\Bbb R^+\;,\;x\in\Bbb R\implies dz=aie^{ix}dx=iz\,dx $$
$$|z-1|^2=|(a\cos x-1)+ai\sin x|^2=a^2-2a\cos x+1\implies$$
$$\int\limits_0^\pi\left(\log(1-2a\cos x+a^2)\right)^2dx=\frac{1}{ai}\int\limits_{\Gamma:=\{|z|=a\;,\;\Im(z)\ge 0\}}\left(\log|z-1|^2\right)^2\,\frac{1}{z}\,dz=$$
$$=-\frac{4i}{a}\int\limits_\Gamma \frac{\log^2|z-1|}{z}\;dz\;\ldots$$
A: If $a$ is real-valued and $|a| >1$, $$\sum_{n=1}^{\infty} \frac{\cos (n \theta)}{n} \left(\frac{1}{a} \right)^{n} = - \frac{1}{2} \log \left(1- \frac{2}{a} \cos \theta+ \frac{1}{a^{2}} \right) = \log|a|  - \frac{1}{2} \log \left(1-2 a \cos \theta +a^{2} \right). $$
This identity can be derived from the Maclaurin series $$\sum_{n=1}^{\infty} \frac{z^{n}}{n} = - \log(1-z) \ , \ |z| <1$$ by replacing $z$ with $\displaystyle \frac{e^{i \theta}}{a}$ and equating the real parts on both sides.
Now rearrange the identity, square both sides, and integrate.
$$ \begin{align}  \int_{0}^{\pi} \log^{2}(1-2a \cos \theta+a^{2}) \ d \theta  &= 4 \log^{2}|a|  \int_{0}^{\pi} d \theta -4 \log|a| \int_{0}^{\pi} \sum_{n=1}^{\infty} \frac{\cos (n \theta)}{n} \left(\frac{1}{a} \right)^{n} \ d \theta \\ &+ 4 \int_{0}^{\pi} \sum_{k=1}^{\infty} \frac{\cos(k \theta)}{k} \left(\frac{1}{a} \right)^{k} \sum_{n=1}^{\infty} \frac{\cos(n \theta)}{n} \left(\frac{1}{a} \right)^{n} \ d \theta \\ &= 4 \pi \log^{2}|a| - 4 \log|a| \sum_{n=1}^{\infty} \frac{1}{n} \left(\frac{1}{a} \right)^{n} \int_{0}^{\pi} \cos (n \theta) \ d \theta \\ &+ 4 \sum_{k=1}^{\infty} \sum_{n=1}^{\infty} \frac{1}{kn} \left(\frac{1}{a} \right)^{k+n} \int_{0}^{\pi} \cos(k \theta) \cos(n \theta) \  d \theta \\ &= 4 \pi \log^{2}|a| - 0 + 4 \sum_{n=1}^{\infty} \frac{1}{n^{2}} \left(\frac{1}{a} \right)^{2n} \int_{0}^{\pi} \cos^{2}(n \theta) \ d \theta  \qquad (1) \\ &= 4 \pi \log^{2}|a| + 2 \pi \sum_{n=1}^{\infty} \frac{1}{n^{2}} \left( \frac{1}{a^{2}}\right)^{n} \\ &= 4 \pi \log^{2}|a| + 2 \pi \ \text{Li}_{2} \left(\frac{1}{a^{2}} \right) \end{align}$$
$(1)$ $\displaystyle \int_{0}^{\pi} \cos(k\theta) \cos(n \theta) \ d \theta = 0$ unless $k=n$
A: Here is an idea I received from a friend. 
Use $\displaystyle 1/2\int_{0}^{2\pi}\left[\log(a-e^{ix})+\log(a-e^{-ix})\right]^{2}dx$
Then, use Gauss's Mean Value Theorem: $\displaystyle 2\pi f(a)=\int_{0}^{2\pi}f(a-e^{\pm ix})dx$ with
$f(z)=\log^{2}(z)$.
This means that $\displaystyle 1/2\int_{0}^{2\pi}\log(a-e^{ix})dx+1/2\int_{0}^{2\pi}\log(a-e^{-ix})dx=2\pi \log^{2}(a)$
The only thing to evaluate is $\displaystyle \int_{0}^{2\pi}\log(a-e^{ix})\log(a-e^{-ix})dx...(1)$
This can be done by factoring out an 'a' inside the parentheses and writing as a sum of two log terms. 
$$\left(\log(a)+\log(1-1/ae^{ix})\right)\left(\log(a)+\log(1-1/ae^{-ix})\right)$$
But note that, due to Gauss MVT, $\displaystyle \int_{0}^{2\pi}\log^{2}(1-1/ae^{\pm ix})dx=\int_{0}^{2\pi}\log(1-1/ae^{\pm ix})dx=0$
Expand and use the series for $\log(1-z)$ to create a double sum that can be written as one sum via Parseval.
$$\sum_{n=1}^{\infty}\frac{e^{ixn}}{na^{n}}\sum_{k=1}^{\infty}\frac{e^{-kix}}{ka^{k}}$$
$$\sum_{n=1}^{\infty}\sum_{k=1}^{\infty}\frac{1}{nka^{k+n}}\int_{0}^{2\pi}e^{i(n-k)x}dx$$
$$2\pi\sum_{n=1}^{\infty}\frac{1}{n^{2}a^{2n}}$$
Note how it falls into place quite easily because the resulting integral evaluates to $\displaystyle \lim_{k\to n}\int_{0}^{2\pi}e^{i(n-k)x}dx=2\pi$
One obtains $$2\pi \sum_{n=1}^{\infty}\frac{1}{n^{2}a^{2n}}=2\pi Li_{2}(1/a^{2})$$, which is added to the $2\pi \log^{2}(a)$ from above and another that is obtained in the expansion of  (1)
A: Let $z=e^{ix}$ and then $\cos x=\frac{1}{2}(e^{ix}+e^{-ix})=\frac{1}{2}\left(z+\frac{1}{z}\right),dz=izdx$. We have
\begin{eqnarray*}
\int_0^\pi(\log(1-2a\cos x+a^2))^2dx&=&\frac{1}{2}\int_{-\pi}^{\pi}(\log(1-2a\cos x+a^2))^2dx\\
&=&\frac{1}{2}\int_{|z|=1}\left[\log(1-a\left(z+\frac{1}{z}\right)+a^2)\right]^2\frac{dx}{iz}
\end{eqnarray*}
Note that
\begin{eqnarray*}
\log(1-a\left(z+\frac{1}{z}\right)+a^2)=\log(1-a)-\log z+\log\left(1+\frac{az(1-az)}{1-a}\right)
\end{eqnarray*}
and $\log z$ is analytic in $|z|<1$ and
$$ \log\left(1+\frac{az(1-az)}{1-a}\right)=\sum_{n=1}^\infty\frac{(-1)^{n-1}}{n}\left(\frac{az(1-az)}{1-a}\right)^n. $$
Thus
\begin{eqnarray*}
\int_0^\pi(\log(1-2a\cos x+a^2))^2dx&=&\frac{1}{2i}2\pi i\text{Res}(\left(\log(1-a\left(z+\frac{1}{z}\right)+a^2)\right)^2,0)\\
&=&\pi\log^2(1-a).
\end{eqnarray*}
