0
$\begingroup$

I am working on a optimization problem which involves the gradient and hessian of $\log(x^TAx)$, where $x$ is an unknown vector and $A$ is a positive definite matrix. How can I derive them? Thanks!

$\endgroup$
1
$\begingroup$

First, let's define a quadratic form and calculate its differential. $$\eqalign{ \alpha &= x^TAx \cr d\alpha &= 2(Ax)^T dx \cr }$$ The objective function is the log of the preceeding, so its differential and gradient are easy to calculate. $$\eqalign{ \lambda &= \log\alpha \cr d\lambda &= \alpha^{-1}d\alpha \cr &= 2\alpha^{-1}(Ax)^T dx \cr g = \frac{\partial\lambda}{\partial x} &= 2\alpha^{-1}Ax \cr }$$ Now let's calculate the differential of $g$ and thence the hessian. $$\eqalign{ dg &= 2\alpha^{-1}A\,dx - 2Ax\,\,\alpha^{-2}\,d\alpha \cr &= 2\alpha^{-1}A\,dx - 4\alpha^{-2}Ax(Ax)^T\,dx \cr H = \frac{\partial g}{\partial x} &= 2\alpha^{-1}A - 4\alpha^{-2}Ax(Ax)^T \cr &= 2\alpha^{-1}A - gg^T \cr }$$

$\endgroup$
  • $\begingroup$ Thank you very much. BTW, A is not necessarily symmetric. Your solution is enough for me to derive the final answer. $\endgroup$ – Bayes Jan 25 at 0:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.