Showing $f(x,y):=(y^2-x^2)(y^2-2x^2)$ has no extrema how can I show that $f(x,y):=(y^2-x^2)(y^2-2x^2)$ has no extrema?
I already computed the first derivatives for $x$ and $y$ but that doesn't really help me either
$f_{1_x}=8x^3-6xy^2$
$f_{2_x}=24x^2-6y^2$
$f_{1_y}=-6x^2y+4y^3$
$f_{2_y}=-6x^2+12y^2$
 A: Your $f(x,y)$ is a homogeneous polynomial of degree $4$. That is,
$$
f(tx,ty)=t^4f(x,y)
$$
for all $x,y,t\in\Bbb{R}$.
It follows immediately that a point $P=(x_0,y_0)$ where $f(P)\neq0$ cannot be an extremal point. This is because $f((1+\epsilon)P)$ and $f((1-\epsilon)P)$ are slightly larger and slightly smaller than $f(P)$ (which is which depends on the sign of $f(P)$).
The function $f$ vanishes along the lines $y=\pm x$ and $y=\pm\sqrt2 x$. Those lines split the plane into eight sectors in such a way that when you move from one sector to an adjacent one the sign of $f(x,y)$ will also change. Therefore $f$ won't have any extrema on those points either.
Observe that no calculus tools such as derivatives are needed to reach this conclusion.
A: We need to find the critical points by


*

*$f_x=-2x(y^2-2x^2)-4x(y^2-x^2)=0$

*$f_y=2y(y^2-2x^2)+2y(y^2-x^2)=0$
which has solution for $x=y=0$ and for $x,y\neq 0$ we have


*

*$f_x=-2x(y^2-2x^2)-4x(y^2-x^2)=0\\\implies -2(y^2-2x^2)-4(y^2-x^2)=-6y^2+8x^2=0\implies3y^2=4x^2$

*$f_y=2y(y^2-2x^2)+2y(y^2-x^2)=0\implies (y^2-2x^2)+(y^2-x^2)=2y^2-3x^2=0$
which has not solutions.
Then note that
$$f(x,y)=(y^2-x^2)(y^2-2x^2)=y^4-3x^2y^2+2x^4=(y^2-\frac32x^2)^2-\frac14x^4$$
and around$(x,y)=(0,0)$ with $f(0,0)=0$ we have that


*

*$f(0,y)=y^4\ge 0$ 

*$f(x,\sqrt{\frac32}\,x)=-\frac14x^4\le 0$ 
