Why is this Taylor Expansion Finite? Let $f$ be bounded with bounded first and second derivatives. Define $F(w)=\int_0 ^wf(t)dt$.
Then using a Taylor expansion, $$\mathbb{E}[F(Y)-F(X)]=\mathbb{E}[(Y-X)f(X)+\frac{1}{2}(Y-X)^2f'(X)+\frac{1}{6}(Y-X)^3f''(X^*)],$$
where $X^*$ is a random quantity in the interval with endpoints $X$ and $Y$.
Why does the Taylor expansion stop at the 3rd term? 
 A: One of the forms of the remainder in Taylor series says (see 1 ) says that under the hypothesis you state, for every $x,y$ there is a number $\zeta$ (that depends on $x,y,F$) between $x$ and $y$ such that the following holds:
$$
F(y) = F(x) + f(x)*(y-x) + \frac{1}{2}f^{'}(x)*(y-x)^2 + \frac{1}{6}f^{"}(\zeta)*(y-x)^2 
.$$
This generalizes the Intermediate Value Theorem.
This is almost what you have, except that if $X,Y$ are random variables (in some probability space), to claim that $\zeta$ is a random variable one would have to show that $\zeta$ can be expressed as a Borel measurable function of $X,Y$, and I don't think that is possible.
The integral form of the remainder can be used to show that:
$$
Remainder = \frac{1}{6}U*(y-x)^2 
$$
where $U$ is a measurable function of $X,Y$ that takes values on the set ${f^"(t*X + (1-t)*Y) | 0 \le t \le 1}$, i.e.: $U$ takes values on the image under $f^"$ of the interval with endpoints $X,Y$.
I don't think one can show that the point can be chosen to be measurable.
This last representation should cover most applications of expressing the remainder as you do.
