What is the $k$th differential of a composite map? Let $V$, $W$, and $X$ be normed linear spaces, and let $A$ be an open subset of $V$. Suppose $F \in C^k(A,W)$ and $G \in C^k(W,X)$. What is $d^k(G\circ F)_\alpha$ (the $k$th differential of $G \circ F$ at $\alpha \in A$)?
For $k=1$, $$d(G\circ F)_\alpha = dG_{F(\alpha)} \circ dF_\alpha.$$
What is the general formula?
 A: I'll do the second derivative without using coordinates
Here's an explanation of the product rule part of this:
What is $dF_\alpha$? For each $\alpha \in A$, it's a linear map from $V \rightarrow W$. So really $dF_-: A \rightarrow \operatorname{Hom}(V,W)$. Its derivative at $\alpha$ is a linear map from $V \rightarrow \operatorname{Hom}(V,W)$ (which is equivalent to a map from $V \times V \rightarrow W$, but this is not how I'll use it). Call it $L_\alpha$ for brevity.
What is $dG_{F(\alpha)}$? Again, for each $\alpha \in A$, it's a linear map from $W \rightarrow X$. So really $dG_{F(-)}: A \rightarrow \operatorname{Hom}(W,X)$. Its derivative at $\alpha$ is a linear map from $V \rightarrow \operatorname{Hom}(W,X)$. Call it $M_\alpha$ for brevity.
What is the composition $dG_{F(\alpha)} \circ dF_{\alpha}$? For each $\alpha \in A$, it's a linear map from $V \rightarrow X$. So really $dG_{F(-)}\circ dF_-: A \rightarrow \operatorname{Hom}(V,X)$. Its derivative at $\alpha$ is a linear map from $V \rightarrow \operatorname{Hom}(V,X)$. Call it $N_\alpha$ for brevity.
Claim: $N_\alpha(v) = M_\alpha(v) \circ dF_\alpha + dG_{F(\alpha)} \circ L_\alpha(v)$.
Note that each of the $\circ$ "products" is $\operatorname{Hom}(W,X) \times \operatorname{Hom}(V,W) \rightarrow \operatorname{Hom}(V,X)$, i.e. simply a composition of linear maps, which in coordinates is a matrix product, and, as the Claim shows, acts as a product here.
Sketch of Proof: The defining property of $L_\alpha$ is that $dF_v = dF_\alpha + L_\alpha(v-\alpha) + o(\|v-\alpha\|)$. The defining property of $M_\alpha$ is similar. If we compose these we get $dG_{F(v)}\circ dF_v = dG_{F(\alpha)}\circ dF_\alpha + dG_{F(\alpha)} \circ L_\alpha(v-\alpha) + M_\alpha(v-\alpha) \circ dF_\alpha + o(\|v-\alpha\|)$, because the term $M_\alpha(F(v)-F(\alpha))\circ L_\alpha(v-\alpha)$ is approximately quadratic in $v-\alpha$, and thus is $o(\|v-\alpha\|)$. This is exactly the defining property for the derivative of $dG_{F(-)}\circ dF_-$, so this proves that the derivative $N_\alpha$ is as stated.
In case you haven't seen little-$o$ notation, the $o(\|v-\alpha\|)$ just refers to terms which are lower order than $v-\alpha$ as $v \rightarrow \alpha$ (e.g. quadratic is lower order than linear for small inputs). See the link for a formal definition.
In some single variable textbooks, the same technique is used for proving the product rule. It's really the same thing that's going on here.

Here's an explanation of the chain rule part of this:
Now $L_\alpha$ is simply the second derivative of $F$, call it $d^2F_\alpha$. To compute $M_\alpha$ in terms of derivatives of $F$ and $G$, use the chain rule. We have $F: A \rightarrow W$ and $dG_-: W \rightarrow \operatorname{Hom}(W,X)$, and $dG_{F(-)}$ is the composition of these maps (instead of the product; see above and compare the difference). The derivative of $F$ at $\alpha$ is $dF_\alpha: V \rightarrow W$ and the derivative of $dG_-$ at $F(\alpha)$ is $d^2G_{F(\alpha)}: W \rightarrow \mathrm{Hom}(W,X)$. So $M_\alpha$ is $d^2G_{F(\alpha)} \circ dF_\alpha : V \rightarrow \mathrm{Hom}(W,X)$.

Overall answer for the second derivative of a composition:
So overall $d^2(G\circ F)_\alpha(v) = (d^2G_{F(\alpha)} \circ dF_\alpha)(v) \circ dF_\alpha + dG_{F(\alpha)} \circ d^2F_\alpha(v)$, with each side of this equation an element of $\operatorname{Hom}(V,X)$.

Comparison with single variable:
In single variable if we have $f$ and $g$ twice-differentiable real-valued maps on $\mathbb{R}$ and a composition $g\circ f$, then we have $(g\circ f)'(x) = g'(f(x)) f'(x)$ and $(g\circ f)''(x) = g''(f(x))f'(x)^2 + g'(f(x))f''(x)$. Notice that this matches the formula above, with $x$ in place of $\alpha$. (There's no mention of $v$ because this is suppressed in single variable calculus and the derivative is simply reported as a scalar, corresponding to either [take your pick; they're equivalent] a the linear map given by multiplication by that scalar, or in coordinates to a $1\times 1$ matrix with that entry, or in coordinates to a single element vector after plugging in the vector $[1]$.)
