Finding $\max$ and min of function I have a problem with following assignment. I need to find critical points of function $$f (x,y)=x^{3}y-3x^{2}y+y^2$$ and determine whether function has min/max or not.
$$Df=[3x^{2}y-6xy,x^3-3x^2+2y]$$
Critical points are:$(0,0)(3,0)(2,2)$ and 
$$D^2f=\begin{pmatrix}6xy-6y &3x^2-6x\\3x^2-6x&2\end{pmatrix}$$
For $(2,2)$ we get positive definite matrix so it has $\min$ in $(2,2)$.
For the other two I have no idea how to examine them. Because matrix is: 
$$\begin{vmatrix}0 & 0 \\ 0 & 2\end{vmatrix}\quad \text{ and } \quad \begin{vmatrix}0 & 0 \\ 0 & 9\end{vmatrix}$$
I will be very glad for help.
 A: Use the  characteristic  polynomial of the Hessian matrix if the roots(eigenvalues) for the polynomial are positive it is positive definite (min) if they are all negative  it is negative definite(max)   if they are negative and positive then its indefinite(saddle point).
A: Here's a hint for how to deal with the point $(0,0)$:
Since the Hessian is only semidefinite, the usual method breaks down, so you'll have to use your imagination and come up with some custom-made method for this particular problem.
The eigenvector with the zero eigenvalue is $\left(\begin{smallmatrix}1\\0\end{smallmatrix}\right)$, and this is the direction where the quadratic form in the Taylor expansion leaves a "hole" where higher-order terms may influence what happens. So a good idea is to look in the $x$ direction.
First we try $f(x,0)$, but that's identically zero, which doesn't lead to any conclusion (other than that if there is a local extremum, it can't be strict).
But what happens if you look at $f(x,x^3)$? (I.e., you look along the curve $y=x^3$, which is tangential to the $x$ axis at the origin.)
