What are higher derivatives?

Question

From Wikipedia:

Higher derivatives can also be defined for functions of several variables, studied in multivariable calculus. In this case, instead of repeatedly applying the derivative, one repeatedly applies partial derivatives with respect to different variables. For example, the second order partial derivatives of a scalar function of n variables can be organized into an n by n matrix, the Hessian matrix. One of the subtle points is that the higher derivatives are not intrinsically defined, and depend on the choice of the coordinates in a complicated fashion (in particular, the Hessian matrix of a function is not a tensor). Nevertheless, higher derivatives have important applications to analysis of local extrema of a function at its critical points. For an advanced application of this analysis to topology of manifolds, see Morse theory.

In multivariable calculus, I was told that higher derivatives were tensors and that was the reason we never went beyond Hessians (none of us had studied tensors before). If higher derivatives aren't tensors, then what are they? Where can I learn more about them?

Answer 1

I have no idea what the Wikipedia entry is trying to say, but higher derivatives are perfectly well-defined intrinsic objects. Let $V=\mathbb R^n$ and $W=\mathbb R^m$ and let $f: V \to W$ be a differentiable function. (This also works for Banach spaces in general, with some small modifications.) The derivative of $f$ is the map $$ Df: V \to L(V,W), $$ where $L(V,W)$ denotes the space of linear transformations, which associates to a point in $V$ the Jacobian derivative of $f$ at that point.

Now considering $L(V,W)$ as a (Banach) space in its own right, $f$ is twice-differentiable when $Df$ is differentiable, in which case the second derivative $D^2f$ of $f$ is the derivative of $Df$, which is then a map $$ D^2f: V \to L(V,L(V,W)). $$ Identifying $L(V,L(V,W))$ with the space of bilinear maps from $V \times V$ to $W$ leads thinking of the second derivative as a $2$-tensor. Clearly this process can be continued, leading to higher-order derivatives as well; in particular, the third derivative of $f$ would be a map $$ D^3f: V \to L(V,L(V,L(V,W))), $$ which can be thought of as a $3$-tensor and some identifications.