better estimate of error using symbol $O$ to replace the little $o$ My book states:

Using symbol $o$, the relation between increment and differential of a
function $f$ which has a derivative at the point $x$ can be written in
the form
$ \begin{equation*} f(x+h) - f(x) = hf'(x) + o(h) \; \text{for} \; h \rightarrow 0 \tag{1} \end{equation*} $
By introducing the symbol $O$, some estimates can now be refined to
indicate a better estimate of the error with the help of it. Thus
given a function $f$ for which $f''$ is defined and continuous.
We have
$ \begin{align} f(x+h) - f(x) = hf'(x) + O(h^2) \; \text{for} \; h \rightarrow 0 \tag{2} \end{align} $

Questions:

*

*Why does (2) hold? Do I have to use Taylor expansion? The book hasn't introduced it yet.

*What's the point of $f''$ being continuous?

*Why using big $O$ we get a better estimate? I guess It is because we are using $h^2$ rather than $h$? For small $h$, $h^2$ gives us more information, see this. However, equation (1) is using $o$ and equation (2) is using $O$, they are not comparable.

Note: This is from Courant's Introduction to calculus and analysis P.254
 A: *

*The next term being of order $h^2$ does ultimately boil down to Taylor expansion. However, you can derive this result without the full force of Taylor expansion using the mean value theorem:

$$f(x+h)=f(x)+h f'(\xi_1)=f(x) + h(f'(x)+(\xi_1-x) f''(\xi_2)) \\
=f(x)+hf'(x)+h(\xi_1-x) f''(\xi_2)$$
where $\xi_1$ is between $x$ and $x+h$ and $\xi_2$ is between $x$ and $\xi_1$.


*Many (but not all) theorems about differentiable functions demand that the derivatives you are using are continuous. Personally I would avoid fretting about the distinction between differentiable and continuously differentiable (or $k$-times differentiable and $k$-times continuously differentiable) functions as a newcomer in analysis. The point in this statement is that if $f$ is twice differentiable everywhere then the error of the linear approximation usually scales like $h^2$ as $h \to 0$, intuitively because it is dominated by the quadratic Taylor term.

*The actual estimate you use involves discarding the error term so in this regard you don't "get a better estimate" of $f(x+h)$ by writing the error term this way. The point is that you have a sharper estimate of the error term itself by writing it as $O(h^2)$ compared to $o(h)$. That's because a $o(h)$ function could be, say, $h^{3/2}$, which goes to zero more slowly than $h^2$.

