Find a continuous function $f$ such $\int_{a}^{a^2+1}f(x)dx=0\quad\forall a\in \Bbb R$ Find  example : exsits a nonzero continuous function $f:\Bbb R\to \Bbb R$ satisfying
$$\int_{a}^{a^2+1}f(x)dx=0\quad\forall a\in \Bbb R$$
the book answer this: $f(x)=x^2(x-1)(x-\frac{3}{5}),0\le x\le 1$,
$f(x)=-f(-x),-1\le x\le 0$. 
$$\int_{-1}^{1}f(x)dx=0$$
But
$$\int_{1/2}^{5/4}f(x)dx\neq 0,a=\dfrac{1}{2}$$
 A: I think the answer you have is the seed of a full answer. You have a formula for $f$ on $[-1,1]$, but then you would need to use the defining relation to extend $f$ to larger and larger domains.
Take $$f(x)=\begin{cases}x^2(x-1)(x-\frac{3}{5})&0\le x\le 1
\end{cases}$$ where we know that with $a=0$, $$\int_0^1 x^2(x-1)(x-\frac{3}{5})=0$$
For $a\in(0,1]$, 
$$\begin{align}
\int_a^{1} x^2(x-1)\left(x-\frac{3}{5}\right)\,dx+\int_1^{a^2+1}f(x)\,dx&\equiv0\quad\text{(apply $\frac{d}{da}$ to both sides of this functional relation)} \\-a^2(a-1)\left(a-\frac{3}{5}\right)+f(a^2+1)\cdot2a&=0\\
f(a^2+1)&=\frac{a}{2}(a-1)\left(a-\frac{3}{5}\right)
\end{align}$$
So for $x\in(1,2]$, using $x=a^2+1$, you have $$f(x)=\frac{\sqrt{x-1}}{2}\left(\sqrt{x-1}-1\right)\left(\sqrt{x-1}-\frac{3}{5}\right)$$ and now you have a formula 
$$f(x)=\begin{cases}x^2(x-1)(x-\frac{3}{5})&0\le x\le 1\\
\frac{\sqrt{x-1}}{2}\left(\sqrt{x-1}-1\right)\left(\sqrt{x-1}-\frac{3}{5}\right)&1<x\leq2
\end{cases}$$ that extends the book's answer to a larger domain. You can continue like this, extending the domain to $(2,5]$, then to $(5,26]$, and so on. For instance, the next extension gives
$$f(x)=\begin{cases}x^2(x-1)(x-\frac{3}{5})&0\le x\le 1\\
\frac{\sqrt{x-1}}{2}\left(\sqrt{x-1}-1\right)\left(\sqrt{x-1}-\frac{3}{5}\right)&1<x\leq2\\
\frac{\sqrt{\sqrt{x-1}-1}}{4\sqrt{x-1}}\left(\sqrt{\sqrt{x-1}-1}-1\right)\left(\sqrt{\sqrt{x-1}-1}-\frac{3}{5}\right)&2<x\leq5
\end{cases}$$ 
Once you've formalized this, check that everything is in order with negative input as well.

This reconciles with your example:
$$\begin{align}
\int_{1/2}^{5/4}f(x)\,dx&=\int_{1/2}^{1}x^2(x-1)(x-\frac{3}{5})\,dx+\int_{1}^{5/4}\frac{\sqrt{x-1}}{2}\left(\sqrt{x-1}-1\right)\left(\sqrt{x-1}-\frac{3}{5}\right)\,dx\\
&=\left[\frac{x^3}{5} -\frac{2 x^4}{5} + \frac{x^5}{5}\right]_{1/2}^1+\left[\frac{1}{5}\left((x-1)^{5/2}-2(x-1)^2+(x-1)^{3/2}\right)\right]_{1}^{5/4}\\
&=-\frac{1}{160}+\frac{1}{160}\\
&=0
\end{align}$$
A: For the information provided, the function $f(x)$ has to be zero if $f\in C^\infty$. Here is a sample proof.
Since $f(x)$ is a smooth function, it can be expanded with Taylor Series
$$f(x)=\sum_{k=0}^{\infty}b_k\,x^k$$
And the integral is calculated as
\begin{align*}
\int_a^{a^2+1}f(x)\,dx&=\sum_{k=0}^\infty b_k\int_a^{a^2+1} x^k\,dx\\
&=\sum_{k=0}^\infty\frac{b_k}{k+1}\left[(a^2+1)^{k+1}-a^{k+1}\right]
\end{align*}
In addition, the term in the square bracket of above expression is definitely positive, because
\begin{align*}
(a^2+1)^n-a^n &= \sum_{i=0}^{n}C_n^ia^{2i}-a^n\\
&=\sum_{i=1}^{n-1}C_n^ia^{2i}+a^{2n}-a^n+1\\
&=\sum_{i=1}^{n-1}C_n^ia^{2i}+(a^n-\frac{1}{2})^2+\frac{3}{4}>0
\end{align*}
for all $n\ge 1$, where $C_n^i$ is the binomial coefficient.
As a consequence, all $b_k$ has to be zero and $f(x)\equiv 0$.
But the discussion is limited to smooth $f(x)$, it is still open to special functions like Weierstrass function and etc.
A: Consider the primitive $$F(x):=\int_0^x f(t)\>dt$$
of such an $f$. It has to be $C^1$, and has to satisfy
$$F(x^2+1)=F(x)\qquad \forall x\in{\mathbb R}\ .\tag{1}$$
Define the sequence $(x_n)_{n\geq0}=(0,1,5,26,677,\ldots)$ recursively by
$$x_0:=0, \qquad x_{n+1}:=x_n^2+1\quad(n\geq0)\ .$$
Then $T:\>x\mapsto x^2+1$ maps the interval $[x_n,x_{n+1}]$ bijectively onto $[x_{n+1},x_{n+2}]$ for each $n\geq0$. Choose a $C^1$-function $\phi:\>[0,1]\to{\mathbb R}$ which is $\equiv0$ in small neighborhoods of $0$ and $1$, and define $F(x)$ for $x\geq0$ recursively as follows:
$$F(x):=\phi(x)\quad (x_0\leq x\leq x_1),\qquad F(x):=F\bigl(\sqrt{x-1}\bigr)\quad(x_n\leq x\leq x_{n+1}, \ n\geq1)\ .$$
In this way $F$ is $C^1$ on all of ${\mathbb R}_{\geq0}$, and satisfies $(1)$ there. Finally put $F(x):=F(x^2+1)$ when $x<0$, and you have an $F$ fulfilling all requirements. The function $f:=F'$ then solves the original problem.
