Most people are familiar with this difference quotient:

$$ f'(x) = \lim_{n \rightarrow \infty} \frac{f(x+h) - f(x)}{h} $$

We require that $f(x)$ is twice differentiable.

The error of that forward difference quotient is, using Taylor:

$$f'(x) = \frac{f(x+h) - f(x)}{h} + + \mathcal{O}(h) $$

Now to the central difference quotient:

We establish that:

$$ f(x+h) = f(x) + hf'(x) + \mathcal{O}(h^2) $$


$$ f(x-h) = f(x) - hf'(x) + \mathcal{O}(h^2) $$

Moving on:

$$ f(x+h) - f(x-h) = 2hf'(x) $$

$$ \frac{f(x+h) - f(x-h)}{2h} = f'(x) $$

Where did I make a mistake ? Because it seems like the error disappeared completely. Seems like we can differentiate any function exactly with this technique ( for any $h$).

So my question is:

  1. How does the error of the forward difference quotient compare to the central difference quotient ?

  2. Also another more general question: What does that error exactly mean ?

I'm a little bit confused because it seems smaller $i$ for $\mathcal{O}(h^i)$ would be better, yet some sources say a bigger $i$ is better.

A short precise sentence about that error term woudl be much appreciated

  • 2
    $\begingroup$ Assuming $f(x+h) = f(x) + hf'(x) + \mathcal{O}(h^2)$ and $f(x-h) = f(x) - hf'(x) + \mathcal{O}(h^2)$ we have $f(x+h)-f(x-h)=2hf'(x)+ \mathcal{O}(h^2).$ Note that $f=\mathcal{O}(h^2)$ and $g=\mathcal{O}(h^2)$ doesn't imply $f-g\equiv 0.$ $\endgroup$ – mfl Aug 14 '18 at 19:55
  • 2
    $\begingroup$ The short answer would be that $\mathcal O(h)$ is not the same thing as $\mathcal O(h)$. $\endgroup$ – Saucy O'Path Aug 14 '18 at 19:58
  • $\begingroup$ my bad, so the error is the same ? why do sources say, the central quotient is better than the forward quotient ? $\endgroup$ – zython Aug 14 '18 at 20:00

I'll try to be as concise as possible. The point is that $O(h^2)$ isn't a "real" algebric term that you can cancel out that simply. I hope this example clarify the big-O notation: call $D_h^Cf(x)$ the centered finite difference approximation, you can say that:

$f'(x)-D_h^Cf(x) = O(h^2)$

This means that exists $h_0$ and $C$ such that for $0<h<h_0$, $ |f'(x)-D_h^Cf(x)| \leq C\cdot h^2$. Concluding, the big-O notation tells you that the difference $f'(x)-D_h^Cf(x)$ is bounded by something which is proportional to $h^2$.

This is the reason why you can't cancel out the $O(h^2)$ in your calculations.

Answering your two questions:

1)The difference between the centered (C) and forward (F) finite difference lies in the fact that those two errors has two different orders, C has order 2, while F has order 1. What does this mean? It means that when $h \rightarrow 0$ (which is generally what we do, we are interested in a small $h$) the C finite difference gives an approximation which is more accurate than the F finite difference. In other terms, if you fix $h$ small (say, $10^{-2}$) the C finite difference is closer to the exact value of $f'(x)$ rather than the F finite difference. It means you can use $h$ which is not ludicrously small to get the same degree of accuracy.

2)That error means how far off you are from the "actual" derivative, using a finite difference. You can think of it geometrically: take a function and draw the tangent to the graph in a point. Then you can construct the approximate tangent using finite difference method. What you find is something similar to the figure: Forward finite difference.

The error tells you, for a given $h$, the discrepancy between the coefficient of the approximated tangent and the actual first derivative, in a given point $x$.

I hope this clarifies.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.