Using higher order derivatives I am currently learning about the general Notion of Differentiability. I came across some difficulties when working with higher order derivatives and I am hoping for confirmation or comments on some questions I have.
In the following, let $E$, $F$ be Banach Spaces, and let $X\subseteq E$ be open.
I do understand that for $x_0\in X$ it is $Df(x_0)\in\mathcal{L}(E,F)$.
My directional derivative for $v\in E\setminus\{0\}$ is defined as the derivative in $0$ of the function $(-\varepsilon,\varepsilon)\to F, t\mapsto f(x_0+tv)$ with $\varepsilon>0$ suitable to keep $x_0\pm\varepsilon v$ in $X$.
So it is $D_vf(x_0)\in\mathcal{L}(\mathbb{R},F)$. When then stating $D_vf(x_0)=Df(x_0)v$ while $Df(x_0)v\in F$, are we already using the identification $\mathcal{L}(\mathbb{R},F)\cong F$?
When extending the notion to higher order derivatives $D^kf(x_0)\in\mathcal{L}^k(E,F)$ I came across the statement $$D^kf(x_0)(h_1,\dots,h_k)=D(\dots D(Df(x_0)h_1)h_2\dots)h_k$$
that should somehow be linked to the above identification and that I really cannot wrap my head around.
It would be nice to see some step-by-step computation of that formula. Should I read myself more into multi-linear maps?
Thanks in advance for any comment.
 A: For the first part of your question (relationship between Gauteaux (directional) and Frechet (overall) derivatives), let's break it down step-by-step. The Frechet derivative 
$$Df:E \rightarrow \mathcal{L}(E,F)$$ 
is a nonlinear function that takes as input a point in $E$ at which to make an affine approximation of $f$, and outputs a linear map from $E$ to $F$ corresponding to this local affine approximation. Therefore, evaluating the Frechet derivative at a point, 
$$Df(x_0)\in\mathcal{L}(E,F),$$ 
yields a linear map from $E$ to $F$. Finally, applying this linear map to a vector,
$$Df(x_0)v \in F,$$
yields a vector in $F$.
On the other hand, the Gauteaux derivative of $f$ at point $x_0$ in direction $v$ is already in $F$, since it is defined by the limit of finite differences, which are in $F$:
$$D_v ~f(x_0) := \lim_{s \rightarrow 0} \frac{\overbrace{f(x_0 + sv)}^{\in F} - \overbrace{f(x_0)}^{\in F}}{s}.$$
So, there is no need to invoke any special identifications here.

For the second part of your question (higher derivatives), it is helpful to keep in mind that the space $\mathcal{L}(X,Y)$ of linear maps from one Banach space $X$ to another $Y$, is itself a Banach space (under the induced operator norm). Therefore the first derivative, 
$$Df:E \rightarrow \mathcal{L}(E,F),$$
is just a nonlinear function mapping between Banach spaces, with the same domain as $f$, but different range (the range being a space of linear operators, $\mathcal{L}(E,F)$). To get the second derivative of $f$, we just take the first derivative of $Df$:
$$D^2 f:= D(Df),$$
$$D^2 f: E \rightarrow \mathcal{L}(E,\mathcal{L}(E,F)).$$
For even higher derivatives you can see where this is going:
\begin{align}
f:&E \rightarrow F \\
Df:&E \rightarrow \mathcal{L}(E, F) \\
D^2f := D(Df):&E \rightarrow \mathcal{L}(E, \mathcal{L}(E, F)) \\
D^3f := D(D^2f):&E \rightarrow \mathcal{L}(E,\mathcal{L}(E, \mathcal{L}(E, F))) \\
D^4f := D(D^3f):&E \rightarrow \mathcal{L}(E,\mathcal{L}(E,\mathcal{L}(E, \mathcal{L}(E, F)))) \\
\dots
\end{align}
The result that prevents this from becoming an ugly mess is the following isomorphism:
$$\mathcal{L}^n(E,\mathcal{L}^m(E,F)) \cong \mathcal{L}^{n+m}(E,F),$$
where $\mathcal{L}^k(X,Y)$ denotes the space of $k$-multilinear functions from $X$ to $Y$ (functions taking in $k$ input vectors from $X$, outputs a vector in $Y$, and is independently linear in each input), and $\cong$ denotes an isometric isomorphism of Banach spaces. Applying this isomorphism recursively allows us to express the higher derivatives in terms of multilinear maps as follows:
\begin{align}
f:&E \rightarrow F \\
Df:&E \rightarrow \mathcal{L}(E, F) \\
D^2f:&E \rightarrow \mathcal{L}^2(E, F) \\
D^3f:&E \rightarrow \mathcal{L}^3(E, F) \\
D^4f:&E \rightarrow \mathcal{L}^4(E,F) \\
\dots
\end{align}
which is much nicer.

A good reference for all this can be found in chapter 9 of the course notes "Methods of Applied Mathematics" by Arbogast and Bona, available online here:
https://www.ma.utexas.edu/users/arbogast/appMath08c.pdf
