Finding error in a an approximation

We want to see the total error in approximating

$$f'(x) \approx \frac{ f(x+h)-f(x) }{h}$$

where $$f: R \to R$$ is differentiable. We can find $$\theta \in [x,x+h]$$ by Taylor's to that

$$f(x+h) = f(x) + f'(x) h + f''( \theta ) h^2 /2$$

If the error in function values is bounded by $$\epsilon$$, prove that the rounding error is bounded by $$2 \epsilon /h$$ and the truncation error is bounded by $$Mh/2$$ where $$M$$ is a bound for $$|f''(t)|$$ for $$t$$ near $$x$$.

Try

We know truncation error is difference between true result and the result that would be produced by algorithm. By the result given above, we see that

$$\frac{ f(x+h) - f(x) }{h} = f'(x) + f''(\theta)h/2$$

So that

$$\underbrace{ \frac{ f(x+h) - f(x) }{h} }_{approx} - \underbrace{f'(x)}_{true \; result} = f''(\theta)h/2$$

So that trucantion error $$E_T$$ is absolute value of the above:

$$E_T = |f''(\theta) | h/2 \leq Mh/2$$

So we have our first result. However, for the rounding error I dont see how it is $$2 \epsilon / h$$. Can someone explain what they really mean? Perhaps am I misunderstanding this part.

The rounding error? Take that upper bound $$\epsilon$$ for the error in function values. We're subtracting two instances of it, which doubles that (at most $$\epsilon - (-\epsilon)$$). Then we divide by $$h$$, for $$\frac{2\epsilon}{h}$$. That's how far our calculated value of the difference quotient $$\frac{f(x+\epsilon)-f(x)}{\epsilon}$$ can be from the true value.
• what do they mean by function values? Are we trying to estimate f(x) exact with $f(x')$ approx.? That is we are given that $|f(x) - f(x') | \leq \epsilon$? and we want to estimate $|f'(x) - f'(x')|$? – James Jan 2 at 7:28
• Say we're trying to calculate some reasonably nice function value - say, $\exp(1.234)$. But this is a computer system - we're looking for an explicit number, and our numbers are represented with finite blocks of memory (likely 64 or 128 bits on a modern system). There's only so many numbers that can be represented exactly that way, and $e^{1.234}$ isn't one of them - so, to get our function value, we'll have to round it off. We'll estimate it as best we can, but we can't possibly do better than the closest number that has a floating-point representation. That's roundoff error. – jmerry Jan 2 at 7:55