18
$\begingroup$

I would like to ask whether my understanding of convexity, Hessian matrix, and positive semidefinite matrix is correct.

For a twice differentiable function $f$, it is convex iff its Hessian $H$ is positive semidefinite.

The Hessian matrix $H$ can be calculated by:

https://en.wikipedia.org/wiki/Hessian_matrix

And the definition of positive semidefinite is:

$$x^THx\geqslant0$$

https://en.wikipedia.org/wiki/Positive-definite_matrix#Negative-definite.2C_semidefinite_and_indefinite_matrices

For example, a function

$$f(x,y)=\frac{x^2}{y^4}$$

where $x\geqslant0, y>0$.

Its Hessian is

$$ \begin{bmatrix} 2y^{-4} & -8xy^{-5}\\ -8xy^{-5} & 20x^2y^{-6}\\ \end{bmatrix} $$

And $$x^THx=6x^2y^{-4}\geqslant0$$

Therefore, $H$ is positive semidefinite and $f(x,y)$ is convex.

On the other hand, the determinant of $H$ is

$$40x^2y^{-10}-64x^2y^{-10}=-24x^2y^{-10}\leqslant0$$

which means $f(x,y)$ is concave.

Since $f(x,y)$ is nonlinear, it cannot be both convex and concave, and there must be something wrong with the derivation above.

I would like to ask which part of my under standing is wrong.

Thank you.

$\endgroup$
3
  • 5
    $\begingroup$ (1) The $H(x,y)$ is positive semidefinite iff $d^T H(x,y) d \ge 0$ for every $d\in\mathbb R^2$. There is no link between $d$ and $x,y$. Using the same variable name with different meaning is a very bad idea. (2) also your test for negative semidefinite respectively for concave is wrong. $\endgroup$
    – user251257
    Mar 19, 2016 at 12:36
  • $\begingroup$ Thank you. Now I understand (1). But for (2), why is it wrong? is the calculation of determinant of $H$ wrong? $\endgroup$
    – Dylan Lan
    Mar 19, 2016 at 13:04
  • 1
    $\begingroup$ For negative definite: the leading minors have alternating signs, starting with negative. Also the diagonal elements need to be negative. Your hessian is indefinite at $x\ne 0$. Positive semidefinite at $x=0$. $\endgroup$
    – user251257
    Mar 19, 2016 at 13:07

2 Answers 2

6
$\begingroup$

I guess the problem is with how you have approached $\vec{x}^{T}H\vec{x} \ge 0$. In this equation, you wish to find whether matrix H is positive definite or not…..

While doing so, the elements of $\vec{x}$ has to be independent of elements of matrix $H$. I mean, when you use this equation to test the definiteness of the matrix, the elements of $H$ are constants and elements of vector $\vec{x}$ are variables. What you have dones is you have took the vector $\vec{x}$ as variable $x$ at position $(1,1)$ and variable $y$ at position $(2,1)$ [mind that the elements of matrix H are also in terms of variables x and y]. If your vector $\vec{x}$ were of other variables, say $\vec{x} = (p,q)$, your answer will be fine.

$\endgroup$
5
$\begingroup$

First, you had $$x^{T}Hx=6x^{2}y^{-4}.$$

$x$ is a non-zero real-valued column vector. Let $$x=\begin{vmatrix} x_{1}\\ x_{2} \end{vmatrix},$$ then $$x^{T}Hx=\begin{vmatrix} x_{1} & x_{2} \end{vmatrix}\begin{vmatrix} 2y^{-4} &-8xy^{-5} \\ -8xy^{-5}& 20x^{2}y^{-6} \end{vmatrix}\begin{vmatrix} x_{1}\\ x_{2} \end{vmatrix}=2x_{1}^{2}y^{-4}+20x_{2}^{2}x^{2}y^{-6}-16x_{1}x_{2}xy^{-5}$$ It is hard to tell whether this function is positive or negative, while the only thing that is known is $$x\geq 0,y> 0.$$

Second, you got the determinant of the Hessian matrix to be $$40x^{2}y^{-10}-64x^{2}y^{-10}=-24x^{2}y^{-10}\leq 0$$ and you concluded that the function was "concave".

While the expression you had for the determinant of the Hessian is correct, your conclusion needs re-considerations.

The determinant of the first principal minor is $$2y^{-4}> 0,$$ since $y>0$.

Then, if the determinant of the Hessian matrix is greater than $0$, then the function is strictly convex. If the determinant of the Hessian is equal to $0$, then the Hessian is positive semi-definite and the function is convex.

For the function in question here, the determinant of the Hessian is $$-24x^{2}y^{-10}\leq 0.$$ There is a lot of uncertainty here. If $x$ takes on the value of $0$ and you have a $0$ determinant for the Hessian, then you have $$\Delta _{1}= 2y^{-4}> 0,\Delta _{2}=-24x^{2}y^{-10}= 0,$$ this is the criterion for positive semi-definite and the function is convex.

However, if $x$ takes on non-zero value, then determinant of the Hessian is indeed negative. In that case, you have $$\Delta _{1}>0,\Delta _{2}< 0.$$ The Hessian matrix is actually indefinite and no conclusion about the concavity (or convexity) of the function can be made from the Hessian matrix.

There are a lot of ambiguities here.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .