Take the matrix $\Sigma\in\mathbb{R}^{n\times n}$ and the function $f:\mathbb{R^n}\rightarrow\mathbb{R}$ in $C^2$.

1) How can I compute the matrix derivation

$$\frac{\partial (tr\left(\Sigma\Sigma^TD^2f\right))}{\partial{\Sigma}}$$

where $tr$ is the trace operator and $D^2f$ the Hessian matrix?

2) How does it simplify, if I choose $\Sigma$ to be a diagonal matrix?

My aim is to see if there can be found a form of the derivative in terms of the Laplacian $\Delta{f}$ and the gradient $\nabla{f}$.


You may be able to use a standard result regarding matrix derivatives:

$$ \frac{\partial Tr(X^TAX)}{\partial X} = (A + A^T)X $$ Separately, the Trace has the property (for appropriate matrix dimensions):

$$ Tr(AB) = Tr(BA) $$

Now, denoting the Hessian of $f$ as $H$, We obtain:

$$ \frac{\partial Tr(\Sigma \Sigma^T H)}{\partial \Sigma } = \frac{\partial Tr(\Sigma^T H\Sigma )}{\partial \Sigma } = (H + H^T)\Sigma = 2H\Sigma $$

Where the last equality follows since the Hessian is symmetric.

For the second part of your question, it should be fairly obvious that if $\Sigma$ is diagonal, then $2H\Sigma $ becomes the Hessian $H$ with each column $h_i$ scaled by a factor of $2\Sigma_{ii}$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.