Prove $\int_0^x \frac{f(u)(x-u)^n}{n!}du=\int_0^x ( \int_0^{u_n}( \dotsb ( \int_0^{u_1}f(t)\,dt ) du_1 ) \dotsb )du_n$ via IBP

Problem 18-22 on p. 327 of Michael Spivak's Calculus (first edition) is

Use induction and integration by parts to show that $$\int_0^x \frac{f(u)(x-u)^n}{n!}du=\int_0^x \left( \int_0^{u_n}\left( \dotsb \left( \int_0^{u_1}f(t)\,dt \right) du_1 \right) \dotsb \right)du_n$$

Previous exercises (14-5 and 14-6) have asked us to prove essentially the same thing by induction and by noting that both sides have the same derivative with respect to $x$ and the same value at zero. So I can see how to do the problem that way.

When I try to solve by integration by parts, I'm getting something funny. The $n=1$ case works out OK. When I try to do $n=2$, for instance, let me show you what I'm getting. I want to show $$\int_0^x \frac{f(u)(x-u)^2}{2!}du = \int_0^x \int_0^{u_2}\int_0^{u_1}f(t)\,dt\,du_1\,du_2.$$ If I substitute into the right hand side, using the $n=1$ case, I get that it suffices to show

$$\int_0^x \frac{f(u)(x-u)^2}{2!}du =\int_0^x \left( \int_0^{u_2}f(u_1)(u_2-u_1)\,du_1 \right)du_2.$$ And if I integrate the LHS by parts, I get that it suffices to show

$$\int_0^x \int_0^u f(t)(x-u)\,dt\,du =\int_0^x \left( \int_0^{u_2}f(u_1)(u_2-u_1)\,du_1 \right)du_2.$$

But note these expressions are not the same.

I could always expand, etc. and show that it comes out right, but I'm trying to find the way by simple integration by parts (that is what the author wants me to see).

I know I'm just missing something simple!

Recall integration by parts:-

$\large \int_{a}^{b} g(u)h(u)du=[g(u)\int h(t)dt]_a^b-\int_a^b\frac{dg(u)}{du}[\int h(t)dt]{du}$

Now, for the integral in question, let

$\large g(u) = \frac{(x-u)^n}{n!}$

$\large h(u) = f(u)$

Integrating by parts we have

$\large \int_0^x\frac{f(u)(x-u)^n}{n!}du=[(\int_0^{u_1}f(t)dt)\frac{(x-u_1)^n}{n!}]_{u_1=0}^{u_1=x}+\int_0^x\frac{(x-u_1)^{(n-1)}}{(n-1)!}[\int_0^{u_1}f(t)dt]du_1$

The first expression on the right hand side is zero, as when $u_1=x$ for the upper limit, the non-integral term is zero, and when $u_1=0$ the integral term is zero as the upper and lower limits are both $0$.

Thus

$\large \int_0^x\frac{f(u)(x-u)^n}{n!}du=\int_0^x\frac{(x-u_1)^{(n-1)}}{(n-1)!}[\int_0^{u_1}f(t)dt]du_1$

which is in line with what you obtain if you set $n$ to 2 (so the LHS of your last two equations, although not appearing to be identical, evaluate to the same result)

Repeating integration by parts on the RHS, we set

$\large g(u)=\frac{(x-u)^{n-1}}{(n-1)!}$

$\large h(u)=\int_0^{u_1}f(t)dt$

This results in

$\large \int_0^x\frac{(x-u_1)^{(n-1)}}{(n-1)!}[\int_0^{u_1}f(t)dt]du_1=[(\int_0^{u_2}(\int_0^{u_1}f(t)dt)du_1)\frac{(x-u_2)^{n-1}}{(n-1)!}]_{u_2=0}^{u_2=x}+\int_0^x\frac{(x-u_2)^{n-2}}{(n-2)!}(\int_0^{u_2}(\int_0^{u_1}f(t)dt)du_1)du_2$

Similar to the reasoning made when the first IBP was carried out, the first term in the RHS is zero, so that we have

$\large \int_0^x\frac{f(u)(x-u)^n}{n!}du=\int_0^x\frac{(x-u_2)^{n-2}}{(n-2)!}(\int_0^{u_2}(\int_0^{u_1}f(t)dt)du_1)du_2$

Iterating again, setting

$\large g(u)=\frac{(x-u)^{n-2}}{(n-2)!}$

$\large h(u)=\int_0^{u_2}(\int_0^{u_1}f(t)dt)du_1$

we have

$\large \int_0^x\frac{f(u)(x-u)^n}{n!}du=\int_0^x\frac{(x-u_3)^{n-3}}{(n-3)!}(\int_0^{u_3}(\int_0^{u_2}(\int_0^{u_1}f(t)dt)du_1)du_2)du_3$

If we were to iterate for the next $(n-3)$ times, we will have

$\large \int_0^x\frac{f(u)(x-u)^n}{n!}du=\int_0^x\frac{(x-u_n)^{0}}{0!}(\int_0^{u_n}\cdots(\int_0^{u_3}(\int_0^{u_2}(\int_0^{u_1}f(t)dt)du_1)du_2)du_3)\cdots du_n$

$\large \int_0^x\frac{f(u)(x-u)^n}{n!}du=\int_0^x(\int_0^{u_n}\cdots(\int_0^{u_3}(\int_0^{u_2}(\int_0^{u_1}f(t)dt)du_1)du_2)du_3)\cdots du_n$