1
$\begingroup$

How do I find the Hessian (second-derivative) of $f(X) = tr(A X^T B X C)$?

Using differential notation, this has the following first differential:

$d tr(A X^T B X C) = tr((CAX^TB + A^T C^T X^T B^T) dX) $.

This straightforwardly leads to the first derivative $(CAX^TB + A^T C^T X^T B^T)^T = B^T X A^TC^T + BXCA$.

If I've done my algebra correctly, the second differential is

$ d^2 tr(A X^T B X C) = tr(CAdX^T B dX + A^TC^TdX^T B^T dX)$.

How do I obtain the Hessian from this? I believe I need to somehow put this into the form $vec(dX)^T H vec(dX)$, from which I can conclude that the Hessian is $H$.

$\endgroup$

1 Answer 1

1
$\begingroup$

$ \def\LR#1{\left(#1\right)} \def\op#1{\operatorname{#1}} \def\vc#1{\op{vec}\LR{#1}} \def\trace#1{\op{Tr}\LR{#1}} \def\frob#1{\left\| #1 \right\|_F} \def\qiq{\quad\implies\quad} \def\p{\partial} \def\grad#1#2{\frac{\p #1}{\p #2}} $Your gradient calculation is indeed correct. To find the Hessian...

First flatten $X$ and the gradient matrix into vectors $$\eqalign{ x &= \vc{X} \\ G &= B^TXA^TC^T + BXCA \\ g &= \vc{G} \\&= \LR{CA\otimes B^T}x + \LR{A^TC^T\otimes B}x \\ }$$ Then calculate the gradient of the gradient (i.e. the Hessian) $$\eqalign{ dg &= \LR{CA\otimes B^T}dx + \LR{A^TC^T\otimes B}dx \\ \grad{g}{x} &= \LR{CA\otimes B^T} + \LR{A^TC^T\otimes B} \;\equiv\; Hessian \\ }$$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .