9
$\begingroup$

I would like to ask whether my understanding of convexity, Hessian matrix, and positive semidefinite matrix is correct.

For a twice differentiable function $f$, it is convex iff its Hessian $H$ is positive semidefinite.

The Hessian matrix $H$ can be calculated by:

https://en.wikipedia.org/wiki/Hessian_matrix

And the definition of positive semidefinite is:

$$x^THx\geqslant0$$

https://en.wikipedia.org/wiki/Positive-definite_matrix#Negative-definite.2C_semidefinite_and_indefinite_matrices

For example, a function

$$f(x,y)=\frac{x^2}{y^4}$$

where $x\geqslant0, y>0$.

Its Hessian is

$$ \begin{bmatrix} 2y^{-4} & -8xy^{-5}\\ -8xy^{-5} & 20x^2y^{-6}\\ \end{bmatrix} $$

And $$x^THx=6x^2y^{-4}\geqslant0$$

Therefore, $H$ is positive semidefinite and $f(x,y)$ is convex.

On the other hand, the determinant of $H$ is

$$40x^2y^{-10}-64x^2y^{-10}=-24x^2y^{-10}\leqslant0$$

which means $f(x,y)$ is concave.

Since $f(x,y)$ is nonlinear, it cannot be both convex and concave, and there must be something wrong with the derivation above.

I would like to ask which part of my under standing is wrong.

Thank you.

$\endgroup$
3
  • 2
    $\begingroup$ (1) The $H(x,y)$ is positive semidefinite iff $d^T H(x,y) d \ge 0$ for every $d\in\mathbb R^2$. There is no link between $d$ and $x,y$. Using the same variable name with different meaning is a very bad idea. (2) also your test for negative semidefinite respectively for concave is wrong. $\endgroup$ – user251257 Mar 19 '16 at 12:36
  • $\begingroup$ Thank you. Now I understand (1). But for (2), why is it wrong? is the calculation of determinant of $H$ wrong? $\endgroup$ – Dylan Lan Mar 19 '16 at 13:04
  • $\begingroup$ For negative definite: the leading minors have alternating signs, starting with negative. Also the diagonal elements need to be negative. Your hessian is indefinite at $x\ne 0$. Positive semidefinite at $x=0$. $\endgroup$ – user251257 Mar 19 '16 at 13:07
4
$\begingroup$

I guess the problem is with how you have approached $\vec{x}^{T}H\vec{x} \ge 0$. In this equation, you wish to find whether matrix H is positive definite or not…..

While doing so, the elements of $\vec{x}$ has to be independent of elements of matrix $H$. I mean, when you use this equation to test the definiteness of the matrix, the elements of $H$ are constants and elements of vector $\vec{x}$ are variables. What you have dones is you have took the vector $\vec{x}$ as variable $x$ at position $(1,1)$ and variable $y$ at position $(2,1)$ [mind that the elements of matrix H are also in terms of variables x and y]. If your vector $\vec{x}$ were of other variables, say $\vec{x} = (p,q)$, your answer will be fine.

$\endgroup$
4
$\begingroup$

First, you had $$x^{T}Hx=6x^{2}y^{-4}$$.

$$x$$ is a non-zero real-valued column vector. Let $$x=\begin{vmatrix} x_{1}\\ x_{2} \end{vmatrix}$$, then $$x^{T}Hx=\begin{vmatrix} x_{1} & x_{2} \end{vmatrix}\begin{vmatrix} 2y^{-4} &-8xy^{-5} \\ -8xy^{-5}& 20x^{2}y^{-6} \end{vmatrix}\begin{vmatrix} x_{1}\\ x_{2} \end{vmatrix}=2x_{1}^{2}y^{-4}+20x_{2}^{2}x^{2}y^{-6}-16x_{1}x_{2}xy^{-5}$$ It is hard to tell whether this function is positive or negative, while the only thing that is known is $$x\geq 0,y> 0$$.

Second, you got the determinant of the Hessian matrix to be $$40x^{2}y^{-10}-64x^{2}y^{-10}=-24x^{2}y^{-10}\leq 0$$ and you concluded that the function was "concave".

While the expression you had for the determinant of the Hessian is correct, your conclusion needs re-considerations.

The determinant of the first principal minor is $$2y^{-4}> 0$$, since $$y>0$$.

Then, if the determinant of the Hessian matrix is greater than $$0$$, then the function is strictly convex. If the determinant of the Hessian is equal to $$0$$, then the Hessian is positive semi-definite and the function is convex.

For the function in question here, the determinant of the Hessian is $$-24x^{2}y^{-10}\leq $$. There is a lot of uncertainty here. If $$x$$ takes on the value of $$0$$ and you have a $$0$$ determinant for the Hessian, then you have $$\Delta _{1}= 2y^{-4}> 0,\Delta _{2}=-24x^{2}y^{-10}= 0$$, this is the criteria for positive semi-definite and the function is convex.

However, if $$x$$ takes on non-zero value, then determinant of the Hessian is indeed negative. In that case, you have $$\Delta _{1}>0,\Delta _{2}< 0$$. The Hessian matrix is actually indefinite and no conclusion about the concavity (or convexity) of the function can be made from the Hessian matrix.

There are a lot of ambiguities here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.