Where does "$f(x_0+h)-f(x_0)=f'(x_0)h+E(h)$" come from?

Question

In my text book it says that if a function $f$ is differentiable at $x_0$ if $f(x_0+h)-f(x_0)=f'(x_0)h+E(h)$ but I don't understand where this has come from. I thought for a function $f$ to be differentiable at $x_0$ we need only ensure that $$\lim_{\delta x \rightarrow 0} \frac{f(x_0+\delta x)-f(x_0)}{\delta x}$$ exists. How are the two equivalent?

Also I don't see why we need the error function tacked on why doesn't $f(x_0+h)-f(x_0)=f'(x_0)h$ hold?

In my mind I think $f(x_0+h)-f(x_0)=f'(x_0)h \implies f'(x_0)=\frac{f(x_0+h)-f(x_0)}{h}$ which says that the gradient at the point is the derivative but this is the case so where am I wrong?

Answer 1

It is not true that $f(x_0+h)-f(x_0)=hf'(x_0)$. In fact, $$\lim_{h\to 0}\frac{f(x_0+h)-f(x_0)}{h}=f'(x_0)$$ which is quite different. The latter statement says that given $\epsilon>0$, there exists $h>0$ such that $$\left|\frac{f(x_0+h)-f(x_0)}{h}-f'(x_0)\right|<\epsilon$$ Rearranging, we get that $$-h\epsilon<f(x_0+h)-f(x_0)-hf'(x)<h\epsilon.$$ Let $E(h)=f(x_0+h)-f(x_0)-hf'(x_0)\in(-h\epsilon,h\epsilon)$. Then we have $$f(x_0+h)-f(x_0)=hf'(x_0)+E(h).$$

Answer 2

This is more related to algebraic manipulation rather than any symbolic manipulation related to $\epsilon, \delta$.

We start with the typical definition of derivative of a function $f$ at a point $x_{0}$:

1) A function $f$ defined in a certain neighborhood of $x_{0}$ is said to be differentiable at point $x_{0}$ if the following limit $$\lim_{h \to 0}\frac{f(x_{0} + h) - f(x_{0})}{h}$$ exists. Moreover when this limit exists it is said to be the derivative of $f$ at $x_{0}$ and is denoted by symbol $f'(x_{0})$.

Note that this definition can be converted into an equivalent but slightly complicated form (which is preferred by some book authors) namely:

2) A function $f$ defined in a certain neighborhood $I$ of $x_{0}$ is said to differentiable at point $x_{0}$ if there is a number $A$ (depending on function $f$ and $x_{0}$) such that $$f(x_{0} + h) - f(x_{0}) = Ah + E(h)$$ for all points $x_{0} + h \in I$ and $\lim_{h \to 0}E(h)/h = 0$. Moreover in this case we say that $A$ is the derivative of $f$ at $x_{0}$ and we write $A = f'(x_{0})$.

The equivalence between the two forms is obvious. Suppose $f$ is differentiable at $x_{0}$ according to definition $(1)$ above. Then we have $$f'(x_{0}) = \lim_{h \to 0}\frac{f(x_{0} + h) - f(x_{0})}{h}\tag{a}$$ Defining $$E(h) = f(x_{0} + h) - f(x_{0}) - hf'(x_{0})\tag{b}$$ we can see that $$\lim_{h \to 0}\frac{E(h)}{h} = \lim_{h \to 0}\frac{f(x_{0} + h) - f(x_{0})}{h} - f'(x_{0}) = f'(x_{0}) - f'(x_{0}) = 0\tag{c}$$ and we have the equation $$f(x_{0} + h) - f(x_{0}) = f'(x_{0})h + E(h)\tag{d}$$ with $E(h)/h \to 0$ as $h \to 0$. So $f$ is differentiable at $x_{0}$ according to definition $(2)$ also with derivative $A = f'(x_{0})$.

In almost similar manner it is easy to establish that if $f$ is differentiable at $x_{0}$ with derivative $f'(x_{0})$ according to definition $(2)$ then it is also differentiable at $x_{0}$ with derivative $f'(x_{0})$ according to definition $(1)$.

I think it is better to give a simple example illustrating definition $(2)$ because it is not that common.

Let $f(x) = x^{2}$ and consider $x_{0} = 2$. Then we have $$f(x_{0} + h) - f(x_{0}) = f(2 + h) - f(2) = (2 + h)^{2} - 4 = 4h + h^{2} = Ah + E(h)$$ where $A = 4, E(h) = h^{2}$. Obviously $E(h)/h = h \to 0$ and hence $f'(x_{0}) = f'(2) = A = 4$.