How do I combine "error of order" terms in numerical analysis? Looking at a 2nd-order Taylor series approximation of the function $f$, I have this:
$$f(t_1) = f(t_0) + hf'(t_0) + {h^2\over 2}f''(t_0) + O(h^3)$$
Now say I approximate $f''(t0)$ with a $2$nd-order central difference method:
$$f''(t) = {1\over 2}{f'(t+h) - f'(t-h)\over h} + O(h^2)$$
What's the resulting error of this method?
The naive approach would be to substitute the central difference equation into the Taylor series, giving something like this:
$$f(t_1) = f(t_0) + hf'(t_0) + {h\over 4}(f'(t_0+h)-f'(t_0-h)) + {1\over 2}O(h^4) + O(h^3)$$
Is that plausible? Would error actually decrease (go from $2$nd-order to $4$th-order)?
 A: If we start with the Taylor expansion (I'll change variables here, too many subscripts confuse me):
$$f(x+h)=f(x)+h f^{\prime}(x)+\frac{h^2}{2}f^{\prime\prime}(x)+\frac{h^3}{3!}f^{\prime\prime\prime}(x)+O(h^4)$$
and the derivative of this w.r.t. $h$
$$f^{\prime}(x+h)=f^{\prime}(x)+h f^{\prime\prime}(x)+\frac{h^2}{2}f^{\prime\prime\prime}(x)+O(h^3)$$
and the version of this with $h$ replaced by negative $h$:
$$f^{\prime}(x-h)=f^{\prime}(x)-h f^{\prime\prime}(x)+\frac{h^2}{2}f^{\prime\prime\prime}(x)+O(h^3)$$
subtracting the third expression from the second expression gives
$$f^{\prime}(x+h)-f^{\prime}(x-h)=2h f^{\prime\prime}(x)+O(h^3)$$
and we see that the even powers drop out of this error expansion.
If we solve for $f^{\prime\prime}(x)$ like so:
$$f^{\prime\prime}(x)=\frac{f^{\prime}(x+h)-f^{\prime}(x-h)}{2h}+O(h^2)$$
and substitute in the first expression,
$$f(x+h)=f(x)+h f^{\prime}(x)+\frac{h^2}{2}\left(\frac{f^{\prime}(x+h)-f^{\prime}(x-h)}{2h}+O(h^2)\right)+\frac{h^3}{3!}f^{\prime\prime\prime}(x)+O(h^4)$$
we can take the $O(h^2)$ within the parentheses out as an $O(h^4)$ term:
$$f(x+h)=f(x)+h f^{\prime}(x)+\frac{h}{2}\left(\frac{f^{\prime}(x+h)-f^{\prime}(x-h)}{2}\right)+\frac{h^3}{3!}f^{\prime\prime\prime}(x)+O(h^4)$$
the leading term after the replaced portion is $O(h^3)$, thus simplifying to
$$f(x+h)=f(x)+h f^{\prime}(x)+\frac{h}{2}\left(\frac{f^{\prime}(x+h)-f^{\prime}(x-h)}{2}\right)+O(h^3)$$
and we see that the formula has $O(h^3)$ error: cutting $h$ in half decreases the error by a factor of $2^3=8$.
