$\int _0^1f^2\left(x\right)dx-2\int _0^{\sqrt{3}-1}\:\left(x+1\right)f\left(x\right)dx\:+1=0$ Let $   f  $ be an increasingly continuous function over $[0,1]$ such that:
$$\int _0^1(f\left(x\right))^2dx-2\int _0^{\sqrt{3}-1}\:\left(x+1\right)f\left(x\right)dx\:+1=0$$
Find all functions $f$ with these properties.
Attempt.:
First, the function $f$ is integrable therefore the computations with integrals work there.
Then I did the following:
$$\int _0^1(f\left(x\right))^2dx-2\int _0^{\sqrt{3}-1}\:\left(x+1\right)f\left(x\right)dx\:+1=$$
$$\int _0^{\sqrt{3}-1}(f\left(x\right))^2dx+\int _{\sqrt{3}-1}^1(f\left(x\right))^2dx-2\int _0^{\sqrt{3}-1}\:\left(x+1\right)f\left(x\right)dx\:+1=$$
$$\int _0^{\sqrt{3}-1}\left(f\left(x\right)-\left(x+1\right)\right)^2dx+\int _{\sqrt{3}-1}^1(f\left(x\right))^2dx-\int _0^{\sqrt{3}-1}\left(x+1\right)^2dx+1=0$$
Since
$$-\int _0^{\sqrt{3}-1}\left(x+1\right)^2dx=-\frac{6\sqrt{3}-10}{3}-3+\sqrt{3}$$
Then
$$\int _0^{\sqrt{3}-1}\left(f\left(x\right)dx-\left(x+1\right)\right)^2dx+\int _{\sqrt{3}-1}^1(f\left(x\right))^2dx-\frac{6\sqrt{3}-10}{3}-2+\sqrt{3}=0$$
$$\int _0^{\sqrt{3}-1}\left(f\left(x\right)dx-\left(x+1\right)\right)^2dx+\int _{\sqrt{3}-1}^1(f\left(x\right))^2dx+\frac{-3\sqrt{3}+4}{3}=0$$
How should I continue from there? I got stuck there. I am not sure this is going to lead me somewhere. Maybe I should have started differently.
Has somebody an idea what the functions look like?
 A: Write $\alpha = \sqrt{3}-1$ for brevity, and note that $\int_0^\alpha(1+x)dx = 1$ and $\int_0^\alpha xdx = \int_\alpha^1dx$. Then we have
\begin{eqnarray}
\int_0^1f(x)^2dx &-& 2\int_0^\alpha(1+x)f(x)dx + 1
\\ &=& \int_0^1f(x)^2dx-2\int_0^{\alpha}xf(x)dx -2\int_0^1f(x)dx+2 \int_{\alpha}^1f(x)dx + 1
\\ &=& \int_0^1\left[f(x)-1\right]^2dx  + 2\left[\int_{\alpha}^1f(x)dx - \int_0^{\alpha}xf(x)dx\right] \\
&=& \int_0^1\left[f(x)-1\right]^2dx + 2\int_0^\alpha[f(\alpha )-f(x)]xdx+2\int_\alpha^1[f(x)-f(\alpha)]dx
\end{eqnarray}
Since $f$ is increasing, all integrands are strictly positive, so this can never be zero.
A: Substituting $f(x)=1+g(x)$, your equation becomes
$$\int_0^{\sqrt{3}-1} {\rm d}x \, g(x)\left[g(x)-2x\right] + \int_{\sqrt{3}-1}^1 {\rm d}x \, g(x) \left[g(x)+2\right]=0 \tag{1}$$
where $g(x)$ is continuously increasing on $(0,1)$.
Evidently this equation is fulfilled if $g=0$. However, it would be not increasing. We show that the LHS is always positive if $g\neq 0$. To this end, we want to bound it from below:
$$\text{LHS}(1) > -2g(\sqrt{3}-1) \int_0^{\sqrt{3}-1} {\rm d}x \,x + 2g(\sqrt{3}-1)\int_{\sqrt{3}-1}^1 {\rm d}x \\
=g(\sqrt{3}-1) \left( -(\sqrt{3}-1)^2+2(2-\sqrt{3})  \right) = 0 \, .$$
