Convexity, Hessian matrix, and positive semidefinite matrix

Question

I would like to ask whether my understanding of convexity, Hessian matrix, and positive semidefinite matrix is correct.

For a twice differentiable function $f$, it is convex iff its Hessian $H$ is positive semidefinite.

The Hessian matrix $H$ can be calculated by:

https://en.wikipedia.org/wiki/Hessian_matrix

And the definition of positive semidefinite is:

$$x^THx\geqslant0$$

https://en.wikipedia.org/wiki/Positive-definite_matrix#Negative-definite.2C_semidefinite_and_indefinite_matrices

For example, a function

$$f(x,y)=\frac{x^2}{y^4}$$

where $x\geqslant0, y>0$.

Its Hessian is

$$ \begin{bmatrix} 2y^{-4} & -8xy^{-5}\\ -8xy^{-5} & 20x^2y^{-6}\\ \end{bmatrix} $$

And $$x^THx=6x^2y^{-4}\geqslant0$$

Therefore, $H$ is positive semidefinite and $f(x,y)$ is convex.

On the other hand, the determinant of $H$ is

$$40x^2y^{-10}-64x^2y^{-10}=-24x^2y^{-10}\leqslant0$$

which means $f(x,y)$ is concave.

Since $f(x,y)$ is nonlinear, it cannot be both convex and concave, and there must be something wrong with the derivation above.

I would like to ask which part of my under standing is wrong.

Thank you.

Answer 1

I guess the problem is with how you have approached $\vec{x}^{T}H\vec{x} \ge 0$. In this equation, you wish to find whether matrix H is positive definite or not…..

While doing so, the elements of $\vec{x}$ has to be independent of elements of matrix $H$. I mean, when you use this equation to test the definiteness of the matrix, the elements of $H$ are constants and elements of vector $\vec{x}$ are variables. What you have dones is you have took the vector $\vec{x}$ as variable $x$ at position $(1,1)$ and variable $y$ at position $(2,1)$ [mind that the elements of matrix H are also in terms of variables x and y]. If your vector $\vec{x}$ were of other variables, say $\vec{x} = (p,q)$, your answer will be fine.

Answer 2

First, you had $$x^{T}Hx=6x^{2}y^{-4}$$.

$$x$$ is a non-zero real-valued column vector. Let $$x=\begin{vmatrix} x_{1}\\ x_{2} \end{vmatrix}$$, then $$x^{T}Hx=\begin{vmatrix} x_{1} & x_{2} \end{vmatrix}\begin{vmatrix} 2y^{-4} &-8xy^{-5} \\ -8xy^{-5}& 20x^{2}y^{-6} \end{vmatrix}\begin{vmatrix} x_{1}\\ x_{2} \end{vmatrix}=2x_{1}^{2}y^{-4}+20x_{2}^{2}x^{2}y^{-6}-16x_{1}x_{2}xy^{-5}$$ It is hard to tell whether this function is positive or negative, while the only thing that is known is $$x\geq 0,y> 0$$.

Second, you got the determinant of the Hessian matrix to be $$40x^{2}y^{-10}-64x^{2}y^{-10}=-24x^{2}y^{-10}\leq 0$$ and you concluded that the function was "concave".

While the expression you had for the determinant of the Hessian is correct, your conclusion needs re-considerations.

The determinant of the first principal minor is $$2y^{-4}> 0$$, since $$y>0$$.

Then, if the determinant of the Hessian matrix is greater than $$0$$, then the function is strictly convex. If the determinant of the Hessian is equal to $$0$$, then the Hessian is positive semi-definite and the function is convex.

For the function in question here, the determinant of the Hessian is $$-24x^{2}y^{-10}\leq $$. There is a lot of uncertainty here. If $$x$$ takes on the value of $$0$$ and you have a $$0$$ determinant for the Hessian, then you have $$\Delta _{1}= 2y^{-4}> 0,\Delta _{2}=-24x^{2}y^{-10}= 0$$, this is the criteria for positive semi-definite and the function is convex.

However, if $$x$$ takes on non-zero value, then determinant of the Hessian is indeed negative. In that case, you have $$\Delta _{1}>0,\Delta _{2}< 0$$. The Hessian matrix is actually indefinite and no conclusion about the concavity (or convexity) of the function can be made from the Hessian matrix.

There are a lot of ambiguities here.