Why is error term in the definition of derivative for $\mathbb{R}^n \to \mathbb{R}^m$ of order $o(h)$?

Question

The following intuition is usually supplied for a definition of derivative. We would like to approximate function $f(x)$ near some point $x_0$ with a linear map and we would like to show that as we get closer to $x_0$, this approximation becomes good (for certain notion of good).

The following equation is true in general (by definition of error term $\varepsilon(h)$).

$$ f(x_0+h) - f(x_0) = L(h) + \varepsilon(h) $$

Now, as $L(h)$ and $\varepsilon$ are vectors of $\mathbb{R}^m$, to compare them it is reasonable to use magnitude. So, "intuitively", we would like magnitude of relative error to become small as we approach $x_0$. In other words, we would like:

$$ \lim \limits_{h \to 0} R(h) = \lim \limits_{h \to 0} \left( \frac{|\varepsilon(h)|}{|L(h)|} \right) = 0. $$

Question 1: This expression does not make sense if $L(h) = 0$ which could certainly be a derivative for some function. How can in that case we say that function $0$ approximates function close to $x_0$ if relative error is undefined? Do we use some different criterion? Do we ignore this case?

Now, if we assume limit is defined and exists, we can use that $|L(h)| \leq M|h|$ as it is a linear map $\mathbb{R}^n \to \mathbb{R}^m$. Then, we get the following.

$$ 0 = \lim \limits_{h \to 0} R(h) \geq \frac{1}{M} \lim \limits_{h \to 0} \left( \frac{|\varepsilon(h)|}{|h|} \right)$$

From here it seems to be if we assume (?) limit exists and assume it is zero by out intuition on what good approximation is, we can show that it necessarily means that error is of order $o(h)$.

Question 2: Is this reasoning valid?

Question 3: The other direction -- if error term is $o(h)$ -- does not seem to imply that relative error tends to $0$. So, could it be that derivative is defined, but its relative error compared to non linear term does not vanish? How to interpret this with mindset of derivative being the best linear approximation?

Appreciate your thoughts and comments.

Answer 1

By definition a function is differentiable when the following holds

$$f(x_0+h) - f(x_0) = L(h) + \varepsilon(h)$$

with

• $L(h)=\nabla f(x_0)\cdot h$
• $\varepsilon(h)=o(|h|) \implies \lim \limits_{h \to 0} \frac{|\varepsilon(h)|}{|h|} = 0$

form this definition we can prove that when a function is differentiable it is also continuous and all directional derivatives exist.

Answer 2

Think in the simple case for maps $\mathbb{R} \to \mathbb{R}$. Suppose we're given a differentiable function $f$ and some point $x_0$. Many linear maps will "approximate" $f$ at $x_0$, in the sense that the difference between that linear map and $f$ tends to $0$ as $x \to x_0$. Indeed, any linear map of the form $\alpha(x-x_0) + f(x_0)$ for $\alpha \in \mathbb{R}$ works. But the best linear approximation is the one where the error tends to $0$ much faster than we would expect it to, namely it tends to $0$ like $o(x-x_0)$. This corresponds to a tangent line geometrically.