# Hessian and gradient of matrices

Assuming that $f: R^n \rightarrow R$, $a \in R^n$, $g: R\rightarrow R$, and $h: R^n \rightarrow R$

What are the expressions for

1. $\nabla f(x)$ and $\nabla^2 f(x)$ where $f(x) = g(h(x))$

and

1. $\nabla f(x)$ and $\nabla^2 f(x)$ where $f(x) = g(a^T x)$

Assuming everything is smooth, the first question is just an application of the so-called chain-rule: $$\nabla_xf=g'(h(x))\nabla_xh.$$ Using the product rule and once more the chain rule, one gets: $${\nabla^2}_xf=g''(h(x)){}^\intercal\nabla_{x}h\times\nabla_xh+g'(h(x)){\nabla^2}_xh.$$
The second question is just an application of the first one with $h\colon x\mapsto {}^\intercal ax$ which is linear.
• The set of matrices is no more than $\mathbb{R}^{n\times n}$, so everything works fine. I have the habit to write my transposed on the left, as I reserve exponents on the right only for exponentiation. My point is that the transposition is on $\nabla_xh$ not $g''(h(x))$ which is, in any case, a scalar. Regarding your other question, this is again the chain rule but coordinate-wise as $x\mapsto g'(h(x))\nabla_xh$ has values in $\mathbb{R}^n$ not $\mathbb{R}$. Intuitively, this is also the only thing which makes sense. – C. Falcon Oct 1 '17 at 0:32