Second derivative of a multi-variate composition? This is an extension of a former question (titled "Second derivative of a composition?"). Consider the functions $f:\mathbb{R}^p\to\mathbb{R}^n$ and $g:\mathbb{R}^n\to\mathbb{R}^p$ and define the composition $h(x) = f(g(x))$, so $h:\mathbb{R}^n\to\mathbb{R}^n$.
The gradient (derivative) of $h$ is
\begin{align}
\nabla h(x) = \underbrace{\nabla f(g(x))}_{n\times p} \underbrace{\nabla g(x)}_{p\times n}
\end{align}
I am interested in computing the derivative of the above, denoted by $\nabla^2 h$.
\begin{align}
\nabla^2 h(x) = \nabla(\nabla f(g(x))\nabla g(x))
\end{align}
I know that I need to use the chain rule and that the result will be an $n\times n\times n$ tensor, but I am confused about how to write the terms such that the dimensions agree. I have started to read about multilinear maps but I am not sure how to apply them here.
 A: Yes, it is a $n\times p\times n$ tensor, let's do some calculation. Let's use the notation $$\nabla^2h=(\frac{\partial ^2z_p}{\partial x_i\partial x_j})_{p,i,j},\nabla^2f=(\frac{\partial ^2z_p}{\partial y_k\partial x_l})_{p,k,l},\nabla^2g=(\frac{\partial ^2y_k}{\partial x_i\partial x_j})_{y,i,j}$$
Let $\{x_i\}$ be coordinates of domain of $g$, let $\{y_k\}$ be coordinates of domain of $f$, $\{z_p\}$ be coordinates of range of $f$. Using chain rule gives you:
$$\frac{\partial ^2h_p}{\partial x_i\partial x_j}=\frac{\partial}{\partial x_i}\sum_k\frac{\partial z_p}{\partial y_k}\frac{\partial y_k}{\partial x_j}=\sum_{k,l}\frac{\partial^2z_p}{\partial y_l \partial y_k}\frac{\partial y_l}{\partial x_j}\frac{\partial y_k}{\partial x_j}+\sum_k\frac{\partial z_p}{\partial y_k}\frac{\partial ^2y_k}{\partial x_i \partial x_j}$$
The first term corresponds to $v_l^{\top}\cdot\nabla^2f_p\cdot v_k$, and the second term corresponds to $\sum_ku_k\cdot \nabla^2g_{i,j}$. Here I mean $\nabla^2f_p$ is a matrix when fixed $p$, and $\nabla^2g_{i,j}$ is a vector when fixed $i,j$.
A: Primarily, one should realize that the laplacian is thus going to have components since $f$'s range is multidimensional.
We begin by saying that a function such as $g$ which goes from $\mathbb{R}^n\rightarrow\mathbb{R}^p$ Can be defined as several functions $g^i$ which for every i is orthogonal to each other and for every i, $g^i:\mathbb{R}^n\rightarrow\mathbb{R}$ and therefore is a sum:
$$g=\sum_{i=1}^pg^i\hat{\textbf{e}}_i$$
Also, we state that a function such as $f$ which goes from $\mathbb{R}^p\rightarrow\mathbb{R}^n$ Can be defined as several functions $f^i$ which for every i is orthogonal to each other and for every i, $f^i:\mathbb{R}^p\rightarrow\mathbb{R}$ and therefore is a sum:
$$f=\sum_{i=1}^nf^i\hat{\textbf{e}}_i$$
Thus, we then say that in a cartesian basis:
$$\frac{\partial}{\partial x_i}f^j(g)=\frac{\partial f^j}{\partial g^k}\frac{\partial g^k}{\partial x^i}$$
Where repeated indices is indicative of summation such that:
$$\frac{\partial f^j}{\partial g^k}\frac{\partial g^k}{\partial x^i}=\frac{\partial f^j}{\partial g^1}\frac{\partial g^1}{\partial x^i}+\frac{\partial f^j}{\partial g^1}\frac{\partial g^1}{\partial x^i}+...+\frac{\partial f^j}{\partial g^p}\frac{\partial g^p}{\partial x^i}$$
Then, we take the derivative again to obtain:
$$\frac{\partial}{\partial x^a}\left(\frac{\partial f^j}{\partial g^k}\frac{\partial g^k}{\partial x^i}\right)=\frac{\partial g^k}{\partial x^i}\frac{\partial g^b}{\partial x^a}\frac{\partial^2f^j}{\partial g^b\partial g^k}-\frac{\partial f^j}{\partial g^k}\frac{\partial^2 g^k}{\partial x^i\partial x^\alpha}$$
This will result in the components of a (1,2) tensor in non-curvilinear basis. To turn these components into a tensor, we multiply by the appropiate bases:
$$\nabla_a\nabla_i f^j\hat{\textbf{e}}^a\hat{\textbf{e}}^i\hat{\textbf{e}_j}=\left(\frac{\partial g^k}{\partial x^i}\frac{\partial g^b}{\partial x^a}\frac{\partial^2f^j}{\partial g^b\partial g^k}-\frac{\partial f^j}{\partial g^k}\frac{\partial^2 g^k}{\partial x^i\partial x^\alpha}\right)\hat{\textbf{e}}^a\hat{\textbf{e}}^i\hat{\textbf{e}}_j$$
Checking the dimension of these, it is indeed a $n\times p\times n$ matrix
