I am trying to understand the following definiton.

$f:\mathbb{R}^n \rightarrow \mathbb{R}^m$ . The total derivative of $f$ in point $a$ is the unique linear map $Df|_a$ such that $$\lim_{h \rightarrow 0}\dfrac{f(a+h)-f(a)-Df|_a(h)}{||h||} = 0$$

Could someone explain why this definition works?

-Why should we divide by $||h||$?

-Why is $Df|_a$ linear?

-How should I interpret this linear map $Df|_a$, what is the meaning of the total derivative?

  • 4
    $\begingroup$ If you don't divide by $||h||$ then it will be true for all linear maps and it will just express the fact that $f$ is continuous at $a$. $\endgroup$ – Captain Lama Mar 12 '16 at 13:10

This is one of the most fundamental definitions in all of analysis.

It says that the increment $\Delta f:=f(a+h)-f(h)$ of the function value should in first approximation be a linear function of the increment $h$ attached at the point $a$. In other terms: We want $$f(a+h)-f(a)=Lh +r(h)\qquad(|h|\ll1)\ ,\tag{1}$$ whereby the error $r(h)$ should be smaller by magnitudes than the linear term $Lh$ when $h$ is small. Now in general $|Lh|$ will be of order $|h|$ for certain $h$. This means that we should require that $$\lim_{h\to0}{|r(h)|\over |h|}=0$$ in order to impart any real content to $(1)$. It turns out that this condition determines $L$ uniquely. If it can be satisfied then $f$ is called differentiable at $a$, and one denotes the resulting $L$ by $Df\bigr|_a$, or similar.

  • 1
    $\begingroup$ I think it should be noted that in the OP's definition the existence of the differential is given for granted, which should not be the case. $\endgroup$ – Massimo Ortolano Mar 12 '16 at 19:11

I think it's much easier to understand if you write the definition as "$Df_{|a}$ is the unique linear map $L$ (if it exists) satisfying $f(x) = f(a) + L(x-a) + o(||x-a||)$ when $x\rightarrow a$".

Maybe it's even clearer if you write "$Df_{|a}$ is the linear part of the unique affine map $A$ (if it exists) satisfying $f(x) = A(x) + o(||x-a||)$ when $x\rightarrow a$".

So you can see that $A$ is the best possible affine approximation of $f$ near $a$ (because the error you make by replacing $f$ with $A$ is negligible compared to any affine map), and $Df_{|a}$ is the linear part of this affine approximation.


The division by $\|h\|$ here is exactly analogous with the division by $h$ in the definition of the (standard) derivative of a real-valued function of a real variable:

$\dfrac{df}{dx}=\underset{h\rightarrow 0}{\lim}\dfrac{f(x+h)-f(x)}{h}$

To answer the second question, $Df|_a$ is linear because it satisfies the linearity property, that is it commutes with addition and scalar multiplication. This is a consequence of how it is defined and is not actually the hard to prove (try using the definition to show $Df|_a+Dg|_a=D(f+g)|_a$ and $D(\alpha f)|_a=\alpha Df|_a$ directly; hint, use some linear algebra)

To answer the final question, it is the natural analog of the familiar derivative of a function $f:\mathbb{R} \rightarrow \mathbb{R}$ for the case of a function from $f:\mathbb{R}^n \rightarrow \mathbb{R}^m$.

  • 2
    $\begingroup$ You are correct that the linearity of $Df\mid_a$ can be proved easily from the definition. This is the proof: "The definition requires $Df\mid_a$ to be linear. Ergo, if it exists, it is linear". It is possible to pick non-linear functions with this same property. But the very purpose of the derivative is to be a linear approximation to the original function. So in the definition is intentionally restricted to linear functions. $\endgroup$ – Paul Sinclair Mar 12 '16 at 17:34
  • $\begingroup$ I was specifically referring to how $Df|_a$ is defined (by the determinantal formula, which only works if $f$ is differentiable), it is a consequence of that definition that it is linear. $\endgroup$ – Justin Benfield Mar 12 '16 at 17:38
  • 1
    $\begingroup$ Not sure what you even mean by the "determinantal formula" (the matrix of partial derivatives, maybe?), but the definition is as given in the OP. Everything else is a theorem, not a definition. $\endgroup$ – Paul Sinclair Mar 13 '16 at 0:27
  • $\begingroup$ Yes, I was referring to the matrix of partial derivatives (actually the $\det$ of that matrix, which is the formula). It can just as well be taken as the definition of the total derivative in this case (I think, maybe some other assumptions are needed, but the given definition should be able to derived from it). Looking at it from that formula helps motivate the more abstract definition given to the OP. $\endgroup$ – Justin Benfield Mar 13 '16 at 16:12
  • $\begingroup$ The jacobian is not the same thing as the derivative. It is useful for change of coordinates in integration and a few other applications, but the purpose of the derivative is to act as a linearized version of the original function, and the jacobian is just not up to that task. The matrix itself (as a linear operator) instead of its determinant does serve as a derivative, but is defined in terms of a specific coordinate system. Thus the version given in the OP is preferred, as it does not depend on a coordinate system. $\endgroup$ – Paul Sinclair Mar 13 '16 at 17:57

Just to add a bit to the previous answers, you can check that this definition meets the "usual" requirement for differentiability if $f:\mathbb R\to \mathbb R.$

Take $x=a$. Then, according to the "new" definition, to find the derivative of $f$ at $a$, we seek an

$L(a):\mathcal L(\mathbb R,\mathbb R)\to \mathcal L(\mathbb R,\mathbb R)$ such that

$\lim _{h\to 0}\frac{f(a+h)-f(a)-L(a)h}{h}=0$.

Now, the linear tranformations from $\mathbb R\to \mathbb R$ are of the form $fh=bh$ for some $b\in \mathbb R$, so we have now, with $f=L(a)$

$\lim _{h\to 0}\frac{f(a+h)-f(a)-bh}{h}=0$, which simplifies to

$\lim _{h\to 0}\left ( \frac{f(a+h)-f(a)}{h}-b \right )=0$ so that $b=f'(a)$, that is, $L(a)h=f'(a)h$


I'd like to add my point of view about such matters.

-Why should we divide by $||h||$?

It is because in a single dimension $h$ is a number, and you can divide by it, whereas in multiple dimensions $h$ is a vector, and you cannot divide by a vector.

But there is a little bit more.
Let us assume you already know differentiation in one dimension, and you are looking to generalize the definition of derivative to higher dimensions, that is, for functions like $f:\mathbb{R}^n \rightarrow \mathbb{R}^m$.
One possibility is that you take the definition of derivative in one dimension and just divide by the norm, without further modifications, e.g.:

$$ f^{\prime}(\mathbf{a})=\lim _{{\mathbf{h}} \rightarrow 0} \frac{1}{|{\mathbf{h}}|}(f(\mathbf{a}+\mathbf{h})-f(\mathbf{a})) $$

But this won't work, not even in one dimension. Indeed, depending on the direction you go along for $h$ to approach zero, the limit $f'(a)$ could change its sign.
Rather, if you bring all the stuff on the LHS, you will have that the limit will be zero (here, $[f^{\prime}(\mathbf{a})]$ is the Jacobian):

$$ \lim _{{\mathbf{h}} \rightarrow 0} \frac{1}{|{\mathbf{h}}|}((\mathbf{f}(\mathbf{a}+{\mathbf{h}})-\mathbf{f}(\mathbf{a}))-[f^{\prime}(\mathbf{a})]({\mathbf{h}}))=0 $$

and it won't matter if the limit changes sign approaching from multiple directions.

Note the numerator should go to zero $faster$ than the denominator if we want the limit to be zero.

-Why is $Df|_a$ linear?

It is linear since the whole point of Analysis is replacing nonlinear objects with linear objects (at least locally), for we know almost nothing about nonlinear stuff, while we have a very developed theory for linear structures. See below the answer to your last question for more informations.

-How should I interpret this linear map $Df|_a$, what is the meaning of the total derivative?

Like we anticipated above, the total derivative (or differential) will approximate the function f $linearly$.

That is the proper interpretation of the differential.
f, however, can be highly nonlinear, that is, it will distort the domain in a very different, and much more complicated way, with respect to its linear approximation. Nonetheless, it turns out that if you consider a sufficiently small open ball centered in a , it will still be a $good$ approximation, that is, the manifold representing the graph of f in a small neighbourhood of f(a) will be well-approximated by its tangent (hyper-)plane at that point.

But something interesting can be said even if we consider a bigger neighbourhood. For the sake of simplicity, let us consider some nonlinear $f:\mathbb{R}^2 \rightarrow \mathbb{R}^2$, and let our open ball be the unit circle.

Suppose that our f, being nonlinear, would deform the unit cicle in a very bad way, as depicted here (one the left, the unit circle in the domain, on the right, its image by f):

enter image description here

Now if we replace f by its linear approximation $Df|_0$, not minding the fact that such approximation will worsen as we get far from zero, the image of the unit circle by $Df|_0$ will be an ellipse, since this is the most general way of transforming a circle by the means of a linear mapping.

Hope this helps.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.