7
$\begingroup$

I am reading Rudin and I am very confused what a derivative is now. I used to think a derivative was just the process of taking the limit like this $$\lim_{h\rightarrow 0} \frac{f(x+h)-f(x)}{(x+h)-x}$$

But between Apostol and Rudin, I am confused in what sense total derivatives are derivatives.

Partial derivatives much more resemble the usual derivatives taught in high school

$$f(x,y) = xy$$

$$\frac{\partial f}{\partial x} = y$$

But the Jacobian doesn't resemble this at all. And according to my books it is a linear map.

If derivatives are linear maps, can someone help me see more clearly how my intuitions about simpler derivatives relate to the more complicated forms? I just don't understand where the limits have gone, why its more complex, and why the simpler forms aren't described as linear maps.

Improve this question
$\endgroup$
  • 2
    $\begingroup$ Note: A map $D: V \to V$ is linear if $D(x + y) = D(x) + D(y)$ and $D(cx) = cD(x)$, for $x, y \in V$ and a scalar (number) $c$. $\endgroup$ – pjs36 May 1 '15 at 2:47
  • $\begingroup$ You could alternatively define the derivative of $ f$ at $ x $ to be a matrix. That might seem more concrete to you, and it would be consistent with what you are used to when $ f : \mathbb R \to \mathbb R $ because a $1 \times 1$ matrix is just a scalar. $\endgroup$ – littleO May 1 '15 at 3:45
  • $\begingroup$ Think of a sphere and a tangent plane - that might help you to get a picture of what happens when the dimension increases (though simple models can be misleading too and can mask more complex behaviour e.g. saddle points). $\endgroup$ – Mark Bennet May 1 '15 at 7:49
11
$\begingroup$

A derivative of a function $f: \mathbb{R}^n \rightarrow \mathbb{R}^m$ at $x \in \mathbb{R}^n$ is a linear map $L : \mathbb{R}^n \rightarrow \mathbb{R}^m$ such that

$$\lim_{v \rightarrow 0} \frac{f(x+v)-f(x)-L(v)}{\|v\|} = 0$$

or alternatively $f(x+v) = f(x) + L(v) + o(\|v\|)$, i.e. $f(x)$ is the constant part of $f$ at $x$, and $L$ is the linear part of $f$ at $x$, and everything else is sub-linear.

In the one-dimensional case, the linear map is just multiplication by a scalar, and we call that scalar $f'(x)$, with $L(v) = f'(x)\, v$ for $v \in \mathbb{R}$.

For the case $f: \mathbb{R}^n \rightarrow \mathbb{R}$, the partial derivative at $x$ in the direction $u$ (a unit vector in $\mathbb{R}^n$ would be $L(u) \in \mathbb{R}$.

This generalizes directly to functions $f: X \rightarrow Y$ between Banach spaces $X$ and $Y$ (vector spaces with a norm which are additionally complete [so that limits are well-behaved]).

Improve this answer
$\endgroup$
  • $\begingroup$ What you are describing is the differential of a function, not its derivative (which in the $1$-dimensional case is the map $x \mapsto f'(x)$). Derivatives can be far from linear maps, like $(e^x)' = e^x$... $\endgroup$ – A.P. May 1 '15 at 11:19
  • 3
    $\begingroup$ @A.P. To my understanding, aes' terminology here is fairly standard. The ambiguity that you seem to be pointing out is completely resolved by referring to "the derivative at $x$". You are right that the $x$ dependence can be quite nonlinear, but that has already been taken out of the problem. To illustrate, in your example, the derivative of $e^x$ at $x$ is the function $y \mapsto e^x y$. $\endgroup$ – Ian May 1 '15 at 12:27
  • 1
    $\begingroup$ @A.P. While that perspective is natural from the perspective of elementary calculus, it generalizes badly enough that redefining the terminology is reasonable (and again, rather standard). Here are two particular issues. First, this perspective completely collapses when you move away from Hilbert spaces: when $X$ is a Banach space, the derivative of a function from $X$ to $\mathbb{R}$ is an element of $X^*$, and $X^*$ cannot be identified with $X$. In other words there is no "gradient". $\endgroup$ – Ian May 1 '15 at 13:16
  • 1
    $\begingroup$ @A.P. (Cont.) Second, this perspective is very misleading when working with differential geometry, where it is quite important to properly distinguish between a manifold, its tangent space, and its cotangent space. In short: in a Hilbert space we may identify the derivative map with a vector in the space, but even in this setting we should at least be mindful that an identification is taking place. $\endgroup$ – Ian May 1 '15 at 13:18
  • 2
    $\begingroup$ @A.P. The question is about derivatives, total derivatives, partial derivatives, and the Jacobian matrix. These are all the subject of multivariable calculus, i.e. derivatives of functions between fin. dim. real vector spaces. The function need not be linear but its derivative at a point is. This, even in generalizations, is called a derivative: see here. You are correct that in differential geometry, there is a directly analogous object called the differential (though some say derivative even for this). $\endgroup$ – aes May 2 '15 at 14:12
6
$\begingroup$

If $f:M\to N$ is some (possibly nonlinear) function (here I have in mind a diffeomorphism), then the Jacobian $J_f$ can be viewed as a linear map taking tangent vectors at some point $p \in M$ and returning a tangent vector at $f(p) \in N$.

Let's consider the case where $M$ and $N$ are both $\mathbb R^3$. $f$ is some vector field (or perhaps it could be viewed as a transformation of the vector space), and the Jacobian $J_f$ tells us about directional derivatives of $f$. Given a direction $v$ at a point $p$, $J_f(v) = (v \cdot \nabla) f|_p$.

If $M$ and $N$ are both $\mathbb R^1$, then what do we have? There's only one linearly independent tangent vector at each point, so $J_f$ is uniquely determined by some unit vector $\hat x$ and we get $J_f(\hat x) = \frac{df}{dx} \hat x$. The Jacobian here just tells us how a unit length is stretched or shrunk when we view $f$ as a transformation of the real line.

This is the way in which a Jacobian is a linear map: it tells us how directions in the domain correspond to directions in the range. And even 1d derivatives can be seen in this way. The components of the Jacobian are still the partial derivatives you're familiar with. We're just using those partial derivatives to talk about transformations of directions under some function, some map.

Improve this answer
$\endgroup$
  • $\begingroup$ +1 I really like how you incorporated tangent vectors. Brings in good old diffeogeo. $\endgroup$ – Stan Shunpike May 1 '15 at 6:36
3
$\begingroup$

The simpler form is a linear map. Regardless of the setting, if you have $G : X \to Y$ which is differentiable at $x$, you will have

$$G(y)=G(x)+G'_x(y-x)+o(\| y - x \|)$$

where $G'_x$ is the derivative of $G$ at $x$, which is a linear map from $X$ to $Y$. When $X=Y=\mathbb{R}$, all linear maps are just multiplication by a real number, so derivatives correspond directly to real numbers. When $X=\mathbb{R}^n,Y=\mathbb{R}^m$, we identify $G'_x$ with a matrix, which we call the Jacobian matrix.

Because of the theorem that the Jacobian of a differentiable function from $\mathbb{R}^n$ to $\mathbb{R}^m$ is the matrix of partial derivatives, there is an analogue of the limit formula in $\mathbb{R}^n$, where each entry is the limit of a particular partial derivative. There is no direct analogue simply because there's no way to make sense of division by a vector (in general).

Improve this answer
$\endgroup$
3
$\begingroup$

I'll just answer based on what linearity of a derivative means; the basic derivative is linear, when viewed from the right context.

As I mentioned in the comment, if we have a vector space $V$ (let's say it's over the field of real numbers, $\Bbb R$), then we say a map $T: V \to V$ is linear if, given vectors $f, g \in V$ and a scalar $c \in \Bbb R$, the following hold:

\begin{align*} T(f + g) &= T(f) + T(g) \\ T(cf) &= cT(f). \end{align*}

So, the trick is to think of a function $f: \Bbb R \to \Bbb R$ as belonging to the vector space of functions from $\Bbb R$ to $\Bbb R$ (if you need to, convince yourself that this really is a vector space; we can add them, multiply by scalars, we have a $0$ function, etc).

If we call this vector space of functions $V$, then the derivative \begin{align*}\frac{d}{dx}: V &\to V\\ f &\mapsto \frac{df}{dx} = f' \end{align*}

is a map from $V$ to $V$ (we apply the derivative to a function, and get a function in return), and it satisfies the linearity requirements, as immortalized by the phrase "the derivative of a sum is the sum of derivatives" and the fact that we can "pull constants out" while taking a derivative;

$$\frac{d(f + g)}{dx} = \frac{df}{dx}+\frac{dg}{dx},$$ and

$$\frac{d(cf)}{dx} = c\frac{df}{dx}.$$

Improve this answer
$\endgroup$
  • $\begingroup$ +1 Ah, that makes lots of sense! So is it a mapping between functions in a function space then? Is that true for all derivatives (at least in multivariable vector calc)? $\endgroup$ – Stan Shunpike May 1 '15 at 3:43
  • $\begingroup$ Exactly! Only when you view derivatives as maps on function spaces do they become linear. I'm certainly not a calc guru, but that should definitely be the case with multivariate calc. They may be functions from $\Bbb R^n$ to $\Bbb R^m$, but they're still objects in some function space. $\endgroup$ – pjs36 May 1 '15 at 3:53
  • $\begingroup$ I think this answer is misinterpreting the question. In multivariable calculus, the derivative of $f $ at $ x $ is a linear transformation from $\mathbb R^m $ to $\mathbb R^n $. $\endgroup$ – littleO May 1 '15 at 4:28
  • $\begingroup$ @littleO Are you saying the total derivative is not a map between functions in a function space? $\endgroup$ – Stan Shunpike May 1 '15 at 4:35
  • 2
    $\begingroup$ This is about something different altogether. For $f: \mathbb{R}^n \rightarrow \mathbb{R}^m$, the derivative itself is a linear map from $\mathbb{R}^n \rightarrow \mathbb{R}^m$. But differentiation $\frac{d}{dx}$ is a linear operator (a fancy name for a linear map between function spaces) from the vector space of differentiable functions to the vector space of functions. This is important, but it's something else entirely, and not related to the "total derivative" and "Jacobian" etc that you were asking about. $\endgroup$ – aes May 1 '15 at 4:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.