# A question about derivatives between Euclidean spaces: how should we construct it and interpret its definition?

As it is known from the single-variable calculus, given $$X\subseteq\textbf{R}$$, a function $$f:X\to\textbf{R}$$ and a adherent point $$x_{0}\in X$$ which is also a limit point, we define the derivative of the function $$f$$ at $$x_{0}$$ (if it exists) by the limit \begin{align*} L = \lim_{x\to x_{0};x\neq x_{0}}\frac{f(x) - f(x_{0})}{x - x_{0}} \end{align*}

Based on it, we can extend its definition to functions between Euclidean spaces. More precisely, given a subset $$E\subseteq\textbf{R}^{n}$$, an interior point $$a\in E$$, a function $$f:E\to\textbf{R}^{m}$$ and a linear transformation $$L:\textbf{R}^{n}\to\textbf{R}^{m}$$, we say that $$f$$ is differentiable at $$a$$, whose derivative is $$L$$, if \begin{align*} \lim_{x\to x_{0};x\neq x_{0}}\frac{\|f(x) - f(x_{0}) - L(x-x_{0})\|}{\|x-x_{0}\|} = 0 \end{align*}

From the ''algebraic'' point of view, it is kind of natural to set up this definition. This is because it is equivalent to the single-variable definition. Indeed, based on the assumption of existence of the derivative, we have that \begin{align*} L = \lim_{x\to x_{0};x\neq x_{0}}\frac{f(x) - f(x_{0})}{x - x_{0}} & \Longleftrightarrow \lim_{x\to x_{0};x\neq x_{0}}\left(\frac{f(x) - f(x_{0})}{x - x_{0}} - L\right) = 0\\\\ & \Longleftrightarrow \lim_{x\to x_{0};x\neq x_{0}}\frac{f(x) - f(x_{0}) - L(x-x_{0})}{x-x_{0}} = 0\\\\ & \Longleftrightarrow \lim_{x\to x_{0};x\neq x_{0}}\frac{|f(x) - f(x_{0}) - L(x-x_{0})|}{|x-x_{0}|} = 0 \end{align*}

Moreover, it is customary to say the derivative $$L$$ is the linear transformation which best (linearly) represents the behavior of $$f$$ near $$x_{0}$$. But what about the second, third and $$n$$-th derivative? How should we construct it and interpret it?

Could someone help me to better understand the notion of derivative between euclidean spaces either through analytic or geometric arguments?

I am new to this. Thus any comment, contribution or explanation is welcome.

Pretty much most of the results of single variable calculus can be generalized to higher dimensions, and for that matter, most of the calculus on $$\Bbb{R}^n$$ can easily be generalized to Banach spaces; take a look at Loomis and Sternberg's Advanced Calculus (chapter 3 mainly), or Henri Cartan's Differential Calculus texts.

Let $$V,W$$ be (think finite-dimensional if you wish) Banach spaces over $$\Bbb{R}$$, let $$A \subset V$$ be open and let $$f:A \to W$$ be a map. We say $$f$$ is differentiable at a point $$a \in A$$ if there is a continuous linear map $$L:V \to W$$ such that \begin{align} \lim_{h \to 0}\dfrac{\lVert f(a+h) - f(a) - L(h) \rVert_W}{\lVert h\rVert_V} &= 0. \end{align} In this case, $$L$$ is unique, and we denote it by the symbol $$Df(a)$$, or $$Df_a$$, or $$f'(a)$$, or $$df(a)$$, or $$df_a$$... or any other notation.

Now, if $$f:A \to W$$ is differentiable at every $$a \in A$$, then we get a new map $$Df:A \to \mathcal{L}(V,W)$$, which assigns to each $$a \in A$$, the derivative $$Df_a$$. Note that since $$V$$ and $$W$$ are vector spaces, and $$W$$ is complete, we can equip $$\mathcal{L}(V,W)$$ with a complete norm as well. (In the finite-dimensional case, all norms are equivalent; i.e generate the same topology, and the spaces are always complete, so you can ignore these technical details if you wish).

In other words, $$Df$$ maps the open subset $$A \subset V$$ of a Banach space into the Banach space $$\mathcal{L}(V,W)$$. We can now ask if this map itself is differentiable at a point $$a$$. (using the same definition as above). In this case, we can take the derivative $$D(Df)_a$$. Said differently again, we're considering the map $$g:A \to \mathcal{L}(V,W)$$ given by $$g(a) = Df_a$$, and asking if $$g$$ is differentiable. In this case, we denote \begin{align} Dg_a := D(Df)_a \equiv D^2f_a \end{align} We call $$D^2f_a$$ the second derivative of $$f$$ at $$a$$. What kind of object is it? Well, \begin{align} Dg_a = D^2f_a \in \mathcal{L}(V, \mathcal{L}(V,W)) \end{align} in words it is a linear map from $$V$$ into $$\mathcal{L}(V,W)$$. One can show that there is an (isometric) isomorphism: \begin{align} \mathcal{L}(V, \mathcal{L}(V,W)) \cong \mathcal{L}^2(V;W), \end{align} where the RHS is the (Banach) space of continuous bilinear maps $$V \times V \to W$$. Because of this isomorphism, we usually think of $$D^2f_a$$ as a bilinear map $$V \times V \to W$$, or a "quadratic form". Also, we never explciitly write the isomorphism in the notation; we always just keep in mind that there is an isomorphism, and decipher from context which interpretation is intended.

So, now we have the second derivative $$D^2f:A \to \mathcal{L}^2(V;W)$$. To construct the third derivative, we ask whether this map is differentiable. Then, once again, by composing with appropriate isomorphisms, you'll see that for each $$a \in A$$, the third derivative $$D(D^2f)_a \equiv D^3f_a$$ is a tri-linear, continuous map $$V \times V \times V \to W$$, i.e an element of $$\mathcal{L}^3(V;W)$$. In general, the $$n^{th}$$ derivative (after using all the isomorphisms) will be $$D^nf:A \to \mathcal{L}^n(V;W)$$, so for each $$a \in A$$, $$D^nf_a \in \mathcal{L}^n(V;W)$$ will be an $$n$$-times-multilinear map.

So, really, the definition of second and higher derivatives are the same: to get the second derivative, you differentiate the first derivative. To get the third derivative, you differentiate the second derivative. And so on. The only thing to keep track of is that now the target space for each successive derivative changes. However, if we specialize to the case that $$V = W =\Bbb{R}$$, then for every $$n$$, $$\mathcal{L}^n(\Bbb{R};\Bbb{R}) \cong \Bbb{R}$$, which is why in single variable calculus, we are able to think of higher derivatives at a point as being just numbers. But, if the domain is not $$\Bbb{R}$$ anymore, then in order to keep track of all the various information about the various directions along which the functions can change, we need to invoke much more linear algebra.

Two of the most important results about these higher derivatives are the following:

• With notation as above, for each $$n$$, $$D^nf_a$$ is a symmetric, multilinear map. In other words, for every $$\xi_1, \dots, \xi_n \in V$$, and any permutation $$\sigma: \{1, \dots n\} \to \{1, \dots, n\}$$, we have \begin{align} (D^nf)_a[\xi_{\sigma(1)}, \dots, \xi_{\sigma}(n)] &= (D^nf)_a[\xi_1, \dots, \xi_n]. \end{align} This is the "proper" way of thinking about the classic theorem regarding equality of mixed partial derivatives.

• We also have an analogue of Taylor's theorem \begin{align} f(a+h) &= f(a) + Df_a[h] + \dfrac{1}{2!}D^2f_a[h,h] + \dots + \dfrac{1}{k!}D^kf_a[\underbrace{h, \dots, h}_{\text{k times}}] + o(\lVert h \rVert^{k+1}) \end{align} (see here for a statement and outline of proof of Taylor's theorem, and here for Taylor's theorem with an explicit bound on the remainder term.)

So, even in this multivariable case, the higher derivatives of the function $$f$$ tell you the higher-order approximations to the function. This is of course very similar to the single-variable case.