First Part: (First-order derivative)

Assuming $f$ is an infinitely differential function everywhere, the Taylor series of $f(x + h)$ at $x$ is \begin{align}\tag{1} f(x + h) = f(x) + hf'(x) + \frac{1}{2}h^2f''(\xi) \end{align} where $\xi$ is a number between $x$ and $x+h$.

After rearrangment of terms in (1), we can write $$ f'(x) = \frac{f(x+h) - f(x)}{h} - \frac{1}{2}hf''(\xi). $$

Now, we define a finite difference approximation of $f'(x)$ by $$ f'_h(x) = \frac{f(x+h) - f(x)}{h}, $$ and we express $$ f'(x) = f'_h(x) + E_1 $$ where approximation error $E_1$ satisfy \begin{align} |E_1| &= |- 0.5 hf''(\xi)| \\ &\leq Ch \end{align} assuming $|- 0.5f''(\xi)| \leq C$. Now, using the definition of Big-O notation, we can say \begin{align}\tag{2} f'(x) = f'_h(x) + O(h) \end{align}

This is a very standard result. However, I have a question for clarification.

Question 1: It seems that the constant $C$ can be based on the local behavior of function between $x$ and $x+h$. Can I say that $C$ depends on $h$? Moreover, can I comment on the behavior of $C$ as $h \to 0$?

Second Part: (Square of the first-order derivative)

Using (2), the square of $f'(x)$ can be expressed as $$ \Big(f'(x)\Big)^2 = \Big(f'_h(x) + O(h)\Big)^2 = \Big(f'_h(x)\Big)^2 + 2f'_h(x)O(h) + O(h^2) = \Big(f'_h(x)\Big)^2 + E_2 $$ where the approximation error $E_2$ is $$ E_2 = 2f'_h(x)O(h) + O(h^2). $$ It seems that, $E_2$ depends on the local approximation quantity $f'_h(x)$.

Question 2: How can we estimate the leading order term for the error $E_2$?


1 Answer 1



You don't necessarily change $C$ as $h \to 0$. The constant $C$ is usually take to be a uniform bound of $f'$ in a fixed interval and, although changing this constant according to $h$ can give you better error bounds, it changes nothing with respect to convergence.


If $f'$ is bounded in some interval $[a,b]$ containing $x, x+h$, you can use Lagrange's theorem to get $|f'_h(x)|\leq \|f'\|_{\infty}$ and, therefore, the term $f'_h(x) O(h)$ is in fact $O(h)$. I think this also answers the question about the leading term.

  • $\begingroup$ Thanks for your answer. I have a related doubt. The approximation error $O(h)$ implies that error decays linearly as $h \to 0$. Does it also mean that I can expect linear behaviour only near to zero not otherwise? I mean, I can expect worse error performance far from zero, i.e. when $h$ is large. $\endgroup$
    – hari
    Commented Apr 3, 2019 at 12:50
  • $\begingroup$ Well, that is in fact a limitation of the $O(h)$ notation. However, the error estimates are also valid for large $h$, you just have to write them like formula (1) in your post. $\endgroup$ Commented Apr 3, 2019 at 22:37
  • $\begingroup$ I am sorry, I am still confused. For large $h$, yes, I can write error estimates using formula (1). But, I can't simply say that error will decrease linearly in an interval, suppose $(x+h,x+h+\delta h)$, defined very far from the point $x$, i.e., $h$ is large. This is due to the fact that the behavior of $f''(\xi)$ is unknown. Does this make sense to you? $\endgroup$
    – hari
    Commented Apr 4, 2019 at 8:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .