Finite difference second derivative: Error analysis I want to approximate $f''(x)$ using finite differences. From 
\begin{align*}
f(x+h)=f(x)+f'(x)h+\frac{1}{2}f''(x)h^2+\frac{1}{6}f'''(x)h^3+O(h^4),\\
f(x-h)=f(x)-f'(x)h+\frac{1}{2}f''(x)h^2-\frac{1}{6}f'''(x)h^3+O(h^4),
\end{align*}
we easily obtain
\begin{align}
f''(x)=\frac{f(x+h)-2f(x)+f(x-h)}{h^2}+O(h^2).
\end{align}
Alternatively, we can apply centered finite differences twice with step size $h/2$ and error $O(h^2)$ to write: 
\begin{align}
f''(x) &= \frac{f'(x+h/2)-f'(x-h/2)}{h}+O(h^2)\\
&=\frac{\frac{f(x+h)-f(x)}{h}-\frac{f(x)-f(x-h)}{h}+O(h^2)}{h}+O(h^2)\\
&=\frac{f(x+h)-2f(x)+f(x-h)}{h^2}+O(h)+O(h^2).
\end{align}
I do not understand the inconsistency between the two approaches. Should the $O(h)$ error term above vanish?
 A: You get
$$f''(x)=\frac{\frac{f(x+h)-f(x)}{h}-\frac{f(x)-f(x-h)}{h}+O(h^2)}{h}$$by reasoning that the two approximations to $f'$ both have leading error $O(h^2)$ and therefore their difference will have error $O(h^2)$.
In fact they have the same leading error term which will cancel upon subtraction, and leave you with the result that you were expecting. 
A: The leading error terms  for $f'\left(x+\frac{h}{2}\right)$ and $f'\left(x-\frac{h}{2}\right)$ are $-\frac{h^2}{24}f'''\left(x+\frac{h}{2}\right)$ and $\frac{h^2}{24}f'''\left(x-\frac{h}{2}\right),$ respectively. You can see this, for example, for $f'\left(x+\frac{h}{2}\right)$ by subtracting the Taylor series expansions of $f(x+h)$ and $f(x)$ about $x+(h/2)$ and then dividing the result by $h.$ So, we get
$$\dfrac{f'\left(x+\frac{h}{2}\right)-f'\left(x-\frac{h}{2}\right)}{h} = \dfrac{f(x+h)-2f(x)+f(x-h)}{h^2}+\dfrac{h}{24}\left[f'''\left(x-\frac{h}{2}\right)-f'''\left(x+\frac{h}{2}\right)\right]+\cdots.$$
As you pointed out, it appears like the leading error term is $O(h),$ but if you Taylor expand $f'''\left(x-\frac{h}{2}\right)$ and $f'''\left(x+\frac{h}{2}\right)$ about $x,$ you will see that the term inside the square bracket above simplifies to $-hf''''(x)+\cdots,$ and therefore the leading error term is indeed $O(h^2)$ as expected. 
