My book states:
"It is also true that if $x^*$ is an interior point and:
- a global maximum of $f$ , then $d^2f(x^*)$ is negative semi-definite.
- a global minimum of $f$ , then $d^2f(x^*)$ is positive semi-definite.
But, it is not true that if $x^*$ is a critical point and $d^2f(x^*)$ is negative (positive) semidefinite, then $x^*$ is a local maximum (minimum).
However consider the function: $x^4+x^2-6xy+3y^2$ .
It has a global minimum on $(x,y)=(1,1)$ and $(x,y)=(-1,-1)$ .
However the Hessian at this points, does not seem to be semi-definite.
Can anyone help me?