Calculating $f'(x)$ with $f(x)$ and a relative error? I want to calculate $f'(x)$ using the formula: $$ f'(x) = \frac{f(x+h) - f(x)}{h}$$. Of course the error here is $o(h)$. However, what if in measuring $f(x)$ and $f(x+h)$ I have a relative error of $\epsilon$? What can I say about the error then?
 A: This will depend on the relative error function $\epsilon(x)$:
Set $$A=A(g,x,h)= \frac{g(x+h)-g(x)}{h}$$ 
Then plugin $g(x)=f(x)(1+\epsilon(x))$. You will get something like
$$A - f'(x)= f(x)\epsilon'(x) + f'(x)\epsilon(x) + h(\dots)$$
Where $(\dots)$ contains higher order derivatives of $f$ or $\epsilon.$
A: If you plug $f(x)(1 + \epsilon(x))$ insted of $f(x)$ into
$$
\frac{f(x+h) -f(x)}{h}
$$
you will get
$$
\frac{f(x+h)(1 + \epsilon(x+h)) -f(x)(1 + \epsilon(x))}{h} = 
\frac{f(x+h) - f(x)}{h} + \frac{f(x+h)\epsilon(x+h) -f(x)\epsilon(x)}{h}.
$$
The first term is $O(h)$ (for twice differentiable $f(x)$) off from $f'(x)$ and the second can be estimated as
$$
\left|\frac{f(x+h)\epsilon(x+h) -f(x)\epsilon(x)}{h}\right| \leq 
\frac{|f(x+h)||\epsilon(x+h)| +|f(x)||\epsilon(x)|}{h} \leq
2\frac{\max_x |f(x)| \max_x |\epsilon(x)|}{h}.
$$
The second term is as big as $O(h^{-1})$, so in practice step size $h$ is never taken to be very small. Usually, one tries to keep the second term as big as the truncation error
$$
\frac{M_2 h}{2} \approx 2\frac{M_0 \epsilon}{h}\\
h \approx 2\sqrt{\frac{M_0 \epsilon}{M_2}}.
$$
Here $M_k = \max_x |f^{(k)}(x)|,\quad \epsilon = \max_x |\epsilon(x)|$.
