Find the extremums of complicated function I have a function:
$$f(x,y) = x^2y^3(6-x-y)$$
I found partial derivatives:
$$f_{x}(x,y)^{'} = xy^3(12-3x-2y)=0$$
$$f_{y}(x,y)^{'} = x^2y^2(18-3x-4y)=0$$
And extremums:
$$(0,0);(0,3);(2,0);(2,3)$$
To find maximum and minimum I take second derivative:
$$f_{xx}(x,y)^{''} = 24xy^3-6xy^3-2xy^3=0$$
$$f_{xy}(x,y)^{''} = 36x^2y^2-9x^2y^2-3x^2y^2=0$$
$$f_{yy}(x,y)^{''} = 36x^2y-6x^3y^2-6x^2y=0$$
And you see in cases with coordinates with zeros its not clear how to find out whether they are maximums or minimus. How to do that?
 A: Let me start with that you first order partial derivatives are correct, but solution of system $f_x=0,f_y=0$ are $(2,3), (0,y), (x,0)$.
Finding second derivatives we have
$$ \begin{array}{}
f_{xx}=12y^3-6xy^3-2y^4 \\
f_{xy}=36xy^2-9x^2y^2-8xy^3 \\
f_{yy}=36x^2y-6x^3y-12x^2y^2
\end{array}
$$
Let's denote $\Delta = f_{xx}f_{xy}-f_{yy}^2$, then we have in point $(2,3)$ $\Delta>0$ and therefore maximum here.
For other points is more easy to investigate directly $f$ behaviour.
Let's take
$$\Delta f(0,y) = f(\Delta x, y+ \Delta y)- f(0,y)=\Delta x^2 (y+\Delta y)^2 \left( (6- \Delta x)(y+\Delta y)- (y+\Delta y)^2\right)$$
then, by direct analysis of sign, we have maximum for $y \in (-\infty,0) \cup (6, +\infty)$ and minimum for $y \in (0,6) $. In points $(0,0),(0,6)$ there is no extremum, as $\Delta f(0,y)$ do not keep sign in neigbourhood.
And at last we take $\Delta f(x,0)=(x+ \Delta x)^2 \Delta y^3(6-x-\Delta x- \Delta y)$. Again investigating sign in point $(x,0)$ gives, that there is no extremum.
