$\newcommand\pd{\partial}$ I have read through the beginning of this pdf on Légendre transforms. My problem is when it states:

$$\nabla^2g(p)=\nabla x(p)\cdot\nabla^2f(p)\cdot\nabla x(p)>0.$$

The equality convinces me, I just note the nabla denotes a Jacobian (the "gradient" of a vector function) and nabla square is the Hessian. But how does the equality prove that thing is positive definite? Does it hold in general that if $A,B$ are multipliable square matrices and $B$ is symmetric positive definite in our case $A=\nabla x,B=\nabla^2f$), then $ABA$ is p.d.? How would I prove it?


I read the answer and the problem seems to be a missing transpose on the left gradient. I tried convincing myself of that, but didn't manage. First of all, forget nablas: let me use $H$ for Hessians and $J$ for those "gradients" which are in fact Jacobians. We know $g(p)=g(\nabla f(x(p))$ and $g(p)=J_x(p)=J_x(\nabla f(x(p))$. Call the coordinates $y_j$ to avoid confusion with $x(p)$. Let me use the chain rule to compute second derivatives:

\begin{align*} \frac{\partial^2g}{\partial y_i\partial y_j}(p)={}&\frac{\pd x_j}{\pd y_i}(p)=\sum_{k,\ell=1}^n\frac{\partial x_i}{\partial y_k}(\nabla f(x(p)))\cdot\frac{\pd(\nabla f)_k}{\pd y_\ell}(x(p))\cdot\frac{\pd x_\ell}{\pd y_i}(p)={} \\ {}={}&\sum_{k,\ell=1}^n(J_x(p))_{jk}H_f(x(p))_{k\ell}J_x(p)_{\ell i}=(J_x(p)\cdot H_f(x(p))\cdot J_x(p))_{ji}. \end{align*}

That seems like just the chain rule, and I can't really see where the transpose would appear in those calculations. What am I doing wrong?

  • $\begingroup$ i think the crucial assumption is, that the $f$ is convex and therefore $H(f)\overbrace{=}^{!} \nabla^2 f>0$. then the right hand side constitutes a positive definite bilinear form and everything is fine. $\endgroup$ – tired Sep 30 '15 at 12:25
  • $\begingroup$ @tired I supposed so, but how does the middle Hessian being p.d. imply the product is too? Look at the second question at the end... $\endgroup$ – MickG Sep 30 '15 at 12:27
  • $\begingroup$ There are many notational inconsistency in that pdf. You should probably find something easier to read. $\endgroup$ – user251257 Sep 30 '15 at 12:51
  • 1
    $\begingroup$ Notice that $x(p)$ is the unique solution of $p=\nabla f(x)$. By differentiating the equation, you obtain $id = \nabla^2 f(x(p)) \cdot \nabla x(p)$. Thus the Jacobian of $x$ is the inverse of the Hessian of $f$ which is by assumption positive definite and thus invertible. $\endgroup$ – user251257 Sep 30 '15 at 16:19
  • 2
    $\begingroup$ $\nabla x$ is symmetric. So transpose wouldn't change anything. $\endgroup$ – user251257 Sep 30 '15 at 17:02

We know $\nabla x$ is symmetric since it equals the Hessian of $g$, so:

$$\nabla x(p)\cdot\nabla^2f(p)\cdot\nabla x(p)=\nabla x(p)^T\cdot\nabla^2f(p)\cdot\nabla x(p),$$

but for any matrix $B$ if $A$ is p.d. then $B^TAB$ also is, hence we conclude. Or rather, we reduce the problem to showing $g\in\mathcal{C}^2$, which is done here.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.