A generalization for Serret's integral $\int_0^a \frac{\ln(1+ax)}{1+x^2}dx$ The integral specified in the title appears in Gradshteyn & Ryzhik 4.291.18, also followed by its sister integral:
$$\int_0^a \frac{\ln(1+ax)}{1+x^2}dx=\int_0^a \frac{a\arctan x}{1+ax}dx=\frac12 \arctan a\ln(1+a^2) \tag 1$$
Some application of the above gives us some nice results, such as:
$$\int_0^1\frac{\arctan x}x\ln\left(\frac{(1+x^2)^3}{(1+x)^2}\right)dx=0$$
Above follows by converting $\int_0^1 \frac{\arctan x\ln(1+x^2)}{x}dx$ to a double integral using $(1)$ and then swapping the order of the integrals.
Or another one:
$$\int_0^1 \frac{\arctan x\ln(1+x^2)}{x(1+x)}dx=\frac{\pi^3}{96}-\frac{\pi}{8}\ln^2 2$$

But the question here would be how to prove the result given in $(1)$?
I tried to differentiate under the integral sign with respect to $a$, but something nasty appeared. Are there any other ideas?
 A: Claim: for any $a>0$,
$$\boxed{ I(a)=\int_{0}^{a}\frac{\log(1+ax)}{1+x^2}\,dx = \color{red}{\arctan(a)\log\sqrt{a^2+1}}.} \tag{0}$$
Proof: 
$$ I(a) = \int_{0}^{1}\frac{a\log(1+a^2 x)}{1+a^2 x^2}\,dx,$$
$$I'(a) = \color{blue}{\int_{0}^{1}\frac{2a^2 x}{(1+a^2 x)(1+a^2 x^2)}\,dx} - \color{purple}{ \int_{0}^{1}\frac{(1-a^2 x^2)}{(1+a^2 x^2)^2}\log(1+a^2 x)\,dx }\tag{1}$$
The blue integral is elementary and the purple integral turns into an elementary integral by integration by parts. In particular, the derivatives of both sides of $(0)$ match. To finish the proof, we just have to show that $(0)$ holds at $a=1$. But that is equivalent to
$$ \int_{0}^{\pi/4}\log(1+\tan\theta)\,d\theta = \frac{\pi}{8}\log(2) \tag{2}$$
that is well-known. Another possible approach is to read the LHS of $(0)$ as 
$$ \int_{0}^{a}\frac{\log(1+ax)}{x}\cdot\frac{x}{x^2+1}\,dx $$
and exploit the dilogarithm reflection formulas.
A: Well, we know that:
$$1+x^2=\left(x-i\right)\left(x+i\right)\tag1$$
So, perform partial fraction decomposition:
$$\mathcal{I}\left(x\right):=\int\frac{\ln\left(1+\text{a}\cdot x\right)}{1+x^2}\space\text{d}x=\frac{i}{2}\cdot\left\{\int\frac{\ln\left(1+\text{a}\cdot x\right)}{x+i}\space\text{d}x-\int\frac{\ln\left(1+\text{a}\cdot x\right)}{x-i}\space\text{d}x\right\}\tag2$$
Now, substitute $\text{u}=x+i$ for the left integral:
$$\int\frac{\ln\left(1+\text{a}\cdot x\right)}{x+i}\space\text{d}x=\int\frac{\ln\left(1+\frac{\text{a}\cdot\text{u}}{1-\text{a}i}\right)}{\text{u}}\space\text{d}\text{u}+\ln\left(1-\text{a}i\right)\int\frac{1}{\text{u}}\space\text{d}\text{u}\tag3$$
Now, substitute $\text{v}=\frac{\text{a}\cdot\text{u}}{\text{a}i-1}$ for the left integral:
$$\int\frac{\ln\left(1+\frac{\text{a}\cdot\text{u}}{1-\text{a}i}\right)}{\text{u}}\space\text{d}\text{u}=-\int-\frac{\ln\left(1-\text{v}\right)}{\text{v}}\space\text{d}\text{v}:=\text{C}_1-\text{Li}_2\left(\text{v}\right)\tag4$$
And for the right integral:
$$\ln\left(1-\text{a}i\right)\int\frac{1}{\text{u}}\space\text{d}\text{u}=\ln\left(1-\text{a}i\right)\ln\left|\text{u}\right|+\text{C}_2\tag5$$
Now, for $\int\frac{\ln\left(1+\text{a}\cdot x\right)}{x-i}\space\text{d}x$ it is the same way of proceeding.
A: $$\begin{align}I=\int_0^a\frac{\ln(1+ax)}{1+x^2}dx&=\int_0^a\frac{\ln\frac{1+a^2}{1+at}}{1+t^2}dt\text{  (transform $x=\frac{a-t}{1+at}$)}\\
&=\arctan a\ln(1+a^2)-\int_0^a\frac{\ln(1+at)}{1+t^2}dt\\
&=\arctan a\ln(1+a^2)-I\\
&=\frac12\arctan a\ln(1+a^2)\end{align}
$$
The key point is the mobius transformation.

How do I discover this transformation? By observing the similar technique OP mentioned. We may let $$x=\frac{b_1+b_2t}{c_1+c_2t}$$ and add some restrictions to $b_1,b_2,c_1,c_2$. We hope this transformation preserves the bounds of integral, hence $$0=\frac{b_1+b_2a}{c_1+c_2a}$$ and $$a=\frac{b_1}{c_1}.$$ Also, we can cancel out $b_2$. Three relations are found, then just try to substitute it into the original integral and hope it works. The result is that it really works.
