# Approximation Error in a Finite Difference Approximation of $\Big(f'(x)\Big)^2$

First Part: (First-order derivative)

Assuming $$f$$ is an infinitely differential function everywhere, the Taylor series of $$f(x + h)$$ at $$x$$ is \begin{align}\tag{1} f(x + h) = f(x) + hf'(x) + \frac{1}{2}h^2f''(\xi) \end{align} where $$\xi$$ is a number between $$x$$ and $$x+h$$.

After rearrangment of terms in (1), we can write $$f'(x) = \frac{f(x+h) - f(x)}{h} - \frac{1}{2}hf''(\xi).$$

Now, we define a finite difference approximation of $$f'(x)$$ by $$f'_h(x) = \frac{f(x+h) - f(x)}{h},$$ and we express $$f'(x) = f'_h(x) + E_1$$ where approximation error $$E_1$$ satisfy \begin{align} |E_1| &= |- 0.5 hf''(\xi)| \\ &\leq Ch \end{align} assuming $$|- 0.5f''(\xi)| \leq C$$. Now, using the definition of Big-O notation, we can say \begin{align}\tag{2} f'(x) = f'_h(x) + O(h) \end{align}

This is a very standard result. However, I have a question for clarification.

Question 1: It seems that the constant $$C$$ can be based on the local behavior of function between $$x$$ and $$x+h$$. Can I say that $$C$$ depends on $$h$$? Moreover, can I comment on the behavior of $$C$$ as $$h \to 0$$?

Second Part: (Square of the first-order derivative)

Using (2), the square of $$f'(x)$$ can be expressed as $$\Big(f'(x)\Big)^2 = \Big(f'_h(x) + O(h)\Big)^2 = \Big(f'_h(x)\Big)^2 + 2f'_h(x)O(h) + O(h^2) = \Big(f'_h(x)\Big)^2 + E_2$$ where the approximation error $$E_2$$ is $$E_2 = 2f'_h(x)O(h) + O(h^2).$$ It seems that, $$E_2$$ depends on the local approximation quantity $$f'_h(x)$$.

Question 2: How can we estimate the leading order term for the error $$E_2$$?

1.

You don't necessarily change $$C$$ as $$h \to 0$$. The constant $$C$$ is usually take to be a uniform bound of $$f'$$ in a fixed interval and, although changing this constant according to $$h$$ can give you better error bounds, it changes nothing with respect to convergence.

2.

If $$f'$$ is bounded in some interval $$[a,b]$$ containing $$x, x+h$$, you can use Lagrange's theorem to get $$|f'_h(x)|\leq \|f'\|_{\infty}$$ and, therefore, the term $$f'_h(x) O(h)$$ is in fact $$O(h)$$. I think this also answers the question about the leading term.

• Thanks for your answer. I have a related doubt. The approximation error $O(h)$ implies that error decays linearly as $h \to 0$. Does it also mean that I can expect linear behaviour only near to zero not otherwise? I mean, I can expect worse error performance far from zero, i.e. when $h$ is large.
– hari
Commented Apr 3, 2019 at 12:50
• Well, that is in fact a limitation of the $O(h)$ notation. However, the error estimates are also valid for large $h$, you just have to write them like formula (1) in your post. Commented Apr 3, 2019 at 22:37
• I am sorry, I am still confused. For large $h$, yes, I can write error estimates using formula (1). But, I can't simply say that error will decrease linearly in an interval, suppose $(x+h,x+h+\delta h)$, defined very far from the point $x$, i.e., $h$ is large. This is due to the fact that the behavior of $f''(\xi)$ is unknown. Does this make sense to you?
– hari
Commented Apr 4, 2019 at 8:19