Critical point of a function of two variables $f(x,y)=3x^4 -4x^2y+y^2$: minimum, maximum or saddle? If $f(x,y)=3x^4 -4x^2y+y^2$ then : 
a)$(0,0)$ is local max point
b)$(0,0)$ is local min point
c)$(0,0)$ is saddle point
d) none
I think we find critical points so we must find all point of equation $\nabla f =0$ then $(0,0)$ is critical point but $f_{x,x} = f_{y,y} =f_{x,y}=0$ now  we can't use  this test.
 A: We know that $f(0,0)=0$.  Now search where $f(x,y)>0$.
We have:
$$
y^2-4x^2y+3x^4>0 
$$
with solution: $x^2<y<3x^2$, so for all neighborhoods of $(0,0)$  the function change sign in the neighborhood and $(0,0)$ is a saddle point.
A: It is easy to check that second derivatives are zero at $(0,0)$, so it is a critical point.
We may notice that 
$$f(x,y)=(2x^2-y)^2-x^4=(2x^2-y)^2-(x^2)^2=(x^2-y)(3x^2-y).$$
We immediately see that for $y=x^2$ or $y=3x^2$ we have $f(x,y)=0$, which means that it cannot be strict local minimum/maximum.
If we write $f(x,y)$ in this form it also suggests to look how the function behaves along other curves of the form $y=kx^2$. (Notice that each such curve goes through $(0,0)$).


*

*For $y=2x^2$ we get $f(x,y)=-x^4\le 0$. (And the inequality is strict if $x\ne 0$.)

*For $y=4x^2$ we get $f(x,y)=3x^4\ge 0$. (And the inequality is strict if $x\ne 0$.) 


So it is not an extremum, which means that it is a saddle point.

Notice that from the expression $f(x,y)=(x^2-y)(3x^2-y)$ we can also easily see where the function has positive/negative answer. (This is also says us that it is not an extremum. This argument was used in Emilio Novati's answer.)
We may also notice that $f(x,y)=g(x^2,y)$, where $g(t,y)=3t^2-4ty+y^2$. We could investigate behavior of $g(t,y)$ at $(0,0)$. But it is not immediately clear how this translates to the behavior of $f(x,y)$. (Because we would have to look only on $t\ge0$.)
