How to find $F^{(n+1)}(x)$ if $F(x)=\int_0^x(x-t)^nu(t)dt$? if $F(x)=\int_0^x(x-t)^nu(t)dt$ then find $F^{(n+1)}(x)$
from Leibniz rule $a(x)=0,b(x)=x,a'(x)=0,b'(x)=1$ and $G'(x)=n(x-t)^{n-1}\\$
so
$F^{(1)}(x)=n\int_0^x(x-t)^{n-1}u(t)dt$ and $F^{(n)}(x)=n.(n-1).(n-2)...1\int_0^x(x-t)^{n-n}u(t)dt=n!\int_0^xu(t)dt$. How can I continue from here? is it $F^{(n+1)}(x)=0$ because applying Leibniz rule one more time gives $G'(x)=0$?
 A: You made a mistake in your final integral. The upper end-point's dependence on $x$ means you should get
$$\frac{{\rm d}}{{\rm d} x} \int_0^x u(t) \ {\rm d} t = u(x)$$

In general,
$$\frac{{\rm d}}{{\rm d} x} \int_0^x g(x,t) \ {\rm d} t = \int_0^x \frac{\partial}{\partial x} 
 g(x,t) \ {\rm d} t + \boxed{g(x,x)}$$
But throughout the question, $g(x,t) \propto (x-t)^{n-i}$ meant $g(x,x) = 0$... until $n - i = 0$, when something special happened.
A: Leibniz's rule tells us that we have
$$\frac{d}{dx}\big(\int_{a(x)}^{b(x)}f(x,t)dt\big)=f(x,b(x))\frac{d}{dx}b(x)-f(x,a(x))\frac{d}{dx}a(x)+\int_{a(x)}^{(b(x)}\frac{\partial }{\partial x}f(x,t)dt.$$
So if $F(x)=\int_{0}^{x}(x-t)^{n}u(t)dt$ then
$$F^{(k)}(x)=n(n-1)\cdot\cdot\cdot(n-k+1)\int_{0}^{x}(x-t)^{n-k}u(t)dt$$
for $1\leq k\leq n$. You then correctly found that $F^{(n)}(x)=n!\int_{0}^{x}u(t)dt$. Note that since $a(x)=0$, $b(x)=x$ we have $a'(x)=0$, but $b'(x)=1$ so that the first doesn't vanish. Applying Leibniz's rule to $F^{(n)}(x)$ we obtain
$$F^{(n+1)}(x)=n!u(x)\frac{d}{dx}b(x)-n!u(0)\frac{d}{dx}a(x)+n!\int_{0}^{x}\frac{\partial}{\partial x}u(t)dt$$
$$=n!u(x).$$
