0
$\begingroup$

Assuming that $f: R^n \rightarrow R$, $a \in R^n$, $g: R\rightarrow R$, and $h: R^n \rightarrow R$

What are the expressions for

  1. $\nabla f(x) $ and $\nabla^2 f(x)$ where $f(x) = g(h(x))$

and

  1. $\nabla f(x) $ and $\nabla^2 f(x) $ where $f(x) = g(a^T x)$
$\endgroup$
0
$\begingroup$

Assuming everything is smooth, the first question is just an application of the so-called chain-rule: $$\nabla_xf=g'(h(x))\nabla_xh.$$ Using the product rule and once more the chain rule, one gets: $${\nabla^2}_xf=g''(h(x)){}^\intercal\nabla_{x}h\times\nabla_xh+g'(h(x)){\nabla^2}_xh.$$

The second question is just an application of the first one with $h\colon x\mapsto {}^\intercal ax$ which is linear.

$\endgroup$
  • $\begingroup$ I actually doubted that chain rule can be applied straightforward even here where we are dealing with matrices. Alright. But one question how did you figure out that the 2nd derivative of g in the second expression should be transposed? $\endgroup$ – Soyol Oct 1 '17 at 0:16
  • $\begingroup$ The set of matrices is no more than $\mathbb{R}^{n\times n}$, so everything works fine. I have the habit to write my transposed on the left, as I reserve exponents on the right only for exponentiation. My point is that the transposition is on $\nabla_xh$ not $g''(h(x))$ which is, in any case, a scalar. Regarding your other question, this is again the chain rule but coordinate-wise as $x\mapsto g'(h(x))\nabla_xh$ has values in $\mathbb{R}^n$ not $\mathbb{R}$. Intuitively, this is also the only thing which makes sense. $\endgroup$ – C. Falcon Oct 1 '17 at 0:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.