What is the error of approximation with $f(x_0) + f'(x_0)*h$ In lectures we said that if we approximate $f(x_0 +h)$ with $f(x_0) + f'(x_0)*h$, we get the error $O(h)$ and then we can write : $f(x_0 +h) = f(x_0) + f'(x_0)*h + O(h)$.
So if I understand correctly, the point is only differentiable if $\lim_{h\rightarrow 0}O(h)/h = 0$, and thus if let's say $h <<1$, then $O(h) <<<<1$.
However I still do not really grasp the idea of $O(h)$ and what does the form $f(x_0 +h) = f(x_0) + f'(x_0)*h + O(h)$ actually mean.
Can somebody help me with understanding ?
 A: The big oh notation, if :
$$f(x)=\mathcal{O}(g(x))$$
It means that there exists a constant $M>0$ such that
$$|f(x)|\leq M\cdot g(x) \forall x \geq a $$
Similarly here it just means that the error is of the order of $h$. That is to say that (in your case)
$$\frac{f(x_0 +h)-f(x_0)-h\cdot f'(x_0)}{h}$$ is bounded for all $h\geq a$.
A: $g(x)=f(x_0)+f′(x_0)·x$ is a straight that pass through point $(x_0,f(x_0))$ and is tangent to the curve $f(x)$ at that point.
So, $f(x_0+h)$ with $h\rightarrow 0$ is a point in the curve very close to $x_0$ and the difference $f(x_0+h)-g(x_0)$ trends to zero.
The line $g(x)$ is a very good aproximation in a small range around $x_0$.
The error is $O(h)$ (some number in the same order as $h$, i.e. very small)
If you add this error to $g(x_0)$ you get the true value of $f(x_0+h)$
A: A nice way to visualize what $O(h)$ could look like can be found in the context of infinitely differentiable functions $f\in C^{\infty}(\mathbb{R})$ (Edit: as pointed out in the comments, to ensure that the Taylor series actually converges to the function, we need to look for an analytical $f\in C^{\omega}(\mathbb{R})$). We can construct the Taylor series for $f$ around $x_0$:
$$ f(x_0+h)=f(x_0)+f'(x_0)h+\Bigg{[}\frac{f''(x_0)}{2!}h^2+\cdots+\frac{f^{(n)}(x_0)}{n!}h^n+\cdots\Bigg{]}.$$
Notice how the part in brackets is a polynomial series with degree at least $2$, so that it really acts as your $O(h)$ in this case.
