Generic expression for Hessian Suppose $x\in\mathbb{R}^n$, $s(x)\in\mathbb{R}^n$ and $r(x)\in\mathbb{R}^n$, being $s$ and $r$ smooth mappings. I know the following property holds:
$$ \dfrac{\partial}{\partial x} s(x)^\intercal A \,r(x) = \left[\dfrac{\partial s(x)}{\partial x}\right]^\intercal A \, r(x) + \left[\dfrac{\partial r(x)}{\partial x}\right]^\intercal A^\intercal \, s(x) \quad\quad\text{with $A\in\mathbb{R}^{n\times n}$.}$$
I'm having troubles in finding a generic expression for the Hessian $\mathscr{H}$, i.e.
$$\mathscr{H} = \dfrac{\partial^2}{\partial x^2} s(x)^\intercal A \,r(x)$$
Any help will be much appreciated! Thanks in advance!
Edit: I'm still not sure if the correct definition is
$$\mathscr{H} = \dfrac{\partial^2}{\partial x^2} s(x)^\intercal A \,r(x) \quad \text{or} \quad \mathscr{H} = \dfrac{\partial^2}{\partial x \partial x^{\intercal}} s(x)^\intercal A \,r(x)$$
as in https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf Equation (98).
 A: Assuming that you can calculate the following matrix and (third-order) tensor derivatives of the mappings.
$$
S = \frac{\partial s}{\partial x},
\quad{\mathbb S}= \frac{\partial S}{\partial x}
                = \frac{\partial^2s}{\partial x^2}
\quad\implies\quad
S_{ij}= \frac{\partial s_i}{\partial x_j},
\quad{\mathbb S}_{ijk} 
= \frac{\partial S_{ij}}{\partial x_k}
= \frac{\partial^2s_i}{\partial x_j\partial x_k}
= \frac{\partial^2s_i}{\partial x_k\partial x_j}
\\
R = \frac{\partial r}{\partial x},
\quad{\mathbb R}= \frac{\partial R}{\partial x}
                = \frac{\partial^2r}{\partial x^2}
\quad\implies\quad
R_{ij}= \frac{\partial r_i}{\partial x_j},
\quad{\mathbb R}_{ijk} 
= \frac{\partial R_{ij}}{\partial x_k}
= \frac{\partial^2r_i}{\partial x_j\partial x_k}
= \frac{\partial^2r_i}{\partial x_k\partial x_j}
\\
$$
Find the differential and gradient of the cost function.
$$\eqalign{
\phi &= A:sr^T \\
d\phi
 &= A:(s\,dr^T + ds\,r^T) \\
 &= A^Ts:dr + Ar:ds \\
 &= A^Ts:R\,dx + Ar:S\,dx \\
 &= (R^TA^Ts + S^TAr):dx \\
\frac{\partial \phi}{\partial x} &= R^TA^Ts + S^TAr \;=\; g \\
}$$
which confirms the cookbook result.
Now calculate the gradient of $g$ (aka the hessian of $\phi$).
$$\eqalign{
dg &= dR^TA^Ts + S^TA\,dr + R^TA^Tds + dS^TAr \\
 &= ({\mathbb R}\,dx)^TA^Ts +S^TA(R\,dx) +R^TA^T(S\,dx) +({\mathbb S}\,dx)^TAr \\
 &= (s^TA\,{\mathbb R} + S^TAR + R^TA^TS + r^TA^T{\mathbb S})\,dx \\
\frac{\partial g}{\partial x} 
 &= (s^TA\,{\mathbb R} + S^TAR + R^TA^TS + r^TA^T{\mathbb S}) \;=\; \frac{\partial^2 \phi}{\partial x^2} \;=\; {\scr H} \\
}$$
