I am trying to find the critical points of the function:
$f(x,y)=2x^4-3x^2y+y^2$
and find the Max, Min and saddle points. What I've done so far is:
$f_x=8x^3-6xy=0 , f_y=-3x^2+2y=0 , f_{xx}=24x^2-6y , f_{yy}=2 , f_{xy}=-6x$
So (0,0) is the only critical point. But using the second partial derivative test:
$\Delta(0,0)=f_{xx} . f_{yy} - f^2_{xy}=0$
Which is inconclusive. Without using MATLAB or similar software and based on calculation, how can we determine whether (0,0) is Max, Min or saddle point?
Or the general question is what to do when it's inconclusive (without using software)?