# Numerical differentiation of functions defined by algorithms

Assume we have a function $$f: [a,b] \to \mathbb{R}$$, which is continuous on its domain, and given by some algorithm which allows us to compute however many digits we want (with ways to estimate error) for any point $$x \in [a,b]$$. However, we have no explicit expressions for $$f(x)$$ in terms of known functions, or explicitly given series, definite integrals, etc.

A good example is some kind of iterated mean with no known closed form $$\mu(1,x)$$ (example), they usually have good convergence, but only a handful of them have known closed form (like arithmetic-geometric mean).

Let's assume that the function has many continuous derivatives around a certain point $$a$$ (maybe an infinite number of them).

Then we can define the Taylor series around this point with non-zero radius of convergence:

$$f(x)=f(a)+f'(a) (x-a)+\frac{1}{2} f''(a) (x-a)^2+\dots$$

However, we have no way of explicitly computing derivatives. We have to approximate them numerically.

Finite difference methods are obviously bad for higher derivatives, though the fact that we have a way to compute as many points of $$f(x)$$ as we want with any precision makes this method viable for the first few derivatives.

What to do about higher orders? I don't want to involve interpolation (and it's not needed, again, we can interpolate the function by its defining algorithm well enough). The only other way I know is through Cauchy contour integral, which is valid for analytic functions. Then we will need to do it numerically and also pick a suitable contour.

The main problems here are whether or not we can prove that the functions is analytic as well as the numerical integration itself, since for higher derivatives the integrand becomes oscillatory.

What methods would you suggest for numerical differentiation of such functions?

• en.wikipedia.org/wiki/Numerical_differentiation – Robert Israel Nov 8 '18 at 12:29
• @RobertIsrael, I have read this article, thank you. I know about numerical differentiation in general. Usually the methods are derived for functions given at limited number of points and with limited precision, which is not the case here – Yuriy S Nov 8 '18 at 12:31
• What about using Taylor series arithmetic? It is a known method in automatic/algorithmic differentiation. And divided differences are not that bad if you use some multi-precision number type. Using a second order method (which would be all symmetric methods) for the $k$th derivative is optimal for step size $h\sim\sqrt[2+k]{\epsilon_{work}}$ to get a (relative) error $\epsilon_{target}\sim \sqrt[2+k]{\epsilon_{work}}^2$. Thus to get $d$ accurate digits you need to compute with about $k/2\cdot d$ digits in the algorithm. This is comparable to the effort in the truncated Taylor series approach. – LutzL Nov 8 '18 at 14:01