# How would the double derivative of $f:\mathbb{R}^N \to \mathbb{R}^M$ i.e., $f''$ look like?

The derivative of $f:\mathbb{R}^N \to \mathbb{R}^M$ is of the form $f':\mathbb{R}^N \to \mathbb{R}^{M \times N}$. I'd like to know how the double derivative look like, i.e, how would $f''$ be ? It maps from $\mathbb{R}^N$ to which space?

PS : Please suggest some good references on this topic.

-
Well... applying what you wrote, to $\mathbb R^{M\times N\times N}$. – Did Jun 12 '12 at 6:05
@did : An $M\times N\times N$ matrix to be defined as a linear map we need to define matrix multiplication for matrices of the form $L\times M\times N$. Where can I find this stuff? I wonder which book deals with this kind of stuff? – Rajesh Dachiraju Jun 12 '12 at 6:08
The main stumbling block to seeing how to construct the double derivative from the first might be an insufficient altitude of abstraction. The derivative of $f:V\to W$ is $f\,':V\to\hom(V,W)$ as the answers state, but $\hom(\Bbb R^n,\Bbb R^m)\cong\Bbb R^{m\times n}$ so we can rewrite the derivative as a matrix function. Similarly $\hom(V,\hom(V,W))\cong \hom(V\otimes V,W)$ and $\Bbb R^n\otimes\Bbb R^n\cong \Bbb R^{n\times n}$ (these are tensor products) so the double derivative can be written as $\Bbb R^m\to\Bbb R^{m\times n\times n}$. – anon Jun 12 '12 at 6:35
The derivative of $f$ is a linear map with an $M\times N$ matrix. So the second derivative will be a linear map with an $(M\times N)\times N$ matrix. The thing in the brackets is just "some number". Also you don't need to multiply these things...it is only linear maps to and from the same space that can be composed. – fretty Jun 12 '12 at 9:04

Since you tagged (differential-geometry), let me give a slightly more geometric perspective.

The "derivative" operation in the category of smooth manifolds is one that associates to a smooth map $f:M\to N$ between smooth manifolds $M$ and $N$ the map $Tf$ (or sometimes written as $\mathrm{d}f$ or $\nabla f$, I will not use the notation $\mathrm{d}f$ so as not to confuse with the exterior differentiation) between the smooth manifolds $TM$ and $TN$, where $TM$ denotes the tangent bundle of $M$. A specific property of the map $Tf$ is that restricted on the tangent space of a point $p\in M$, $Tf(p): T_pM \to T_{f(p)}N$ is a linear map between the tangent spaces of $M$ at $p$ and of $N$ and $f(p)$.

This is already slightly different from the calculus definition. For linear spaces $X$ and $Y$, the calculus definition of a derivative gives that for $f:X\to Y$, $\nabla f: X\times X \to Y$ is linear in the second component. Formally we can think of the map $Tf$ as $(f,\nabla f)$ in this context, using that there is a canonical isomorphism between the tangent spaces $TX$ and $X\times X$ when $X$ is linear.

Going back to the geometrical picture: naturally, the "second derivative" then is $$T(Tf) : T(TM) \to T(TN)$$ since the tangent bundles $TM$ and $TN$ are smooth manifolds in their own right, and $Tf$ is a smooth map in its own right. In the advanced calculus definition, given a linear space $X$ and a linear space $Y$, the second derivative is a map from $X\times X\times X \to Y$ that is bilinear in the second and third components. Analogous to the case before, we can try to formally identify $$T(Tf) "=" ((f,\nabla f),\nabla(f,\nabla f)) \overset{?}{=} (f,\nabla f,\nabla f,\nabla^2 f)$$ While the co-domain has the correct number of dimensions, the domain however has some trouble: the domain of $T(Tf)$ is $TTM$, a manifold of $4m$ dimensions (assuming $M$ has $m$ dimensions). $\nabla^2 f$ however takes input as something only $3m$ dimensional! Furthermore, our intuitive notion of the second derivative from advanced calculus consists of taking two derivatives in two possibility different directions, where directions are interpreted as vectors along $M$, that is to say, two objects in $TM$. But as seen above, the notion of $T(Tf)$ requires considering objects in $TTM$. This (at the level of the second derivative) is the point where geometry departs from merely "calculus on manifolds", and this is the point where the notion of linear connection comes in. More precisely, to reconcile the notion with our usual notion of second derivatives, what we require is a way, when given $p\in M$, $v,w\in T_pM$, to identify the triple $(p,v,w)$ with an element of $T_{(p,v)}(TM)$.

For more about these types of stuff, you may wish to consult:

• Kobayashi and Nomizu, Foundations of Differential Geometry
• Kobayashi, S. "Theory of connections" Ann. Mat. Pura Appl. (4), 1957, 43, 119-194
• Kolář, Michor, & Slovák, Natural operations in differential geometry
-

Let $X=\mathbb{R}^n$ and $Y=\mathbb{R}^m$, $f:X\to Y$ then $f': X\to L(X,Y)$ where $L(X,Y)$ is the space of (continuous) linear transformations. Since, by definition, we have $f''=(f')':X\to L(X,L(X,Y))$. Now if L$_2(X,Y)$ is the set of (continuous) bilinear functions $h:X\times X\to Y$ we can identify $L(X,L(X,Y))\cong L_2(X,Y)$ via the mapping $g(x_1)(x_2)\mapsto g(x_1,x_2)$. So the second derivative can be seen as a function $f'':X\to L_2(X,Y)$.

A reference for this would be Jost's "Postmodern analysis".

-

Really $f' : \mathbb R^N \to \mathcal L(\mathbb R^N,\mathbb R^m)$ (linear maps). if $a \in \mathbb R^N$ then $f'(a)$ in a linear map from $\mathbb R^N$ to $\mathbb R^M$. New, since $f''(a)=(f')'(a)$ , we can see that $f'' : \mathbb R^N \to \mathcal L(\mathbb R^N , \mathcal L(\mathbb R^N,\mathbb R^M))$ who is naturally isomorphic to $\mathcal L_2((\mathbb R^N)^2, \mathbb R^M)$ : the set of bilinear maps frome $(\mathbb R^N)^2$ to $\mathbb R^M$.

-