Is $f(x,y) = x^2y + x y^2$ (quasi-) concave or convex?

Question

I should analyze whether the function

$$f(x,y) = x^2y + x y^2 \text{ where } x,y > 0$$

is (quasi-) concave or convex.

Thus, as usual, I set up the Hessian as

$$ D^2f(x,y) = \left( \begin{array}{cc} 2y & 2x + 2y \\ 2x + 2y & 2x \\ \end{array} \right) $$

which, given the constraints $x,y > 0$, is indefinite.

So I need the bordered Hessian:

$$ H= \left( \begin{array}{ccc} 0 & 2xy + y^2 & x^2 + 2xy \\ 2xy + y^2 & 2y & 2x + 2y \\ x^2 + 2xy & 2x+2y & 2x \end{array} \right) $$

and check its determinant to find out about its properties. I end up with

$$ \det|H| = 2x(2x^3 + x^3 y - 2x^2 y + 4x^2 y^2 + 4 x y^3 + 2 y^4)$$

which, considering $x,y > 0$, looks quite positive to me, so $f$ would be quasiconcave. But how can I prove that $\det|H| > 0$? Or is there any easier approach?

Thanks!

Answer 1

The Hessian $D_2$ of your function $f$ is as you said, then: $$ D_2 - \lambda I = \begin{pmatrix} 2y - \lambda & 2x+2y \\ 2x+2y & 2x-\lambda \end{pmatrix}. $$ The determinant is: $$ \det(D_2-\lambda I) = \lambda^2 - 2(x+y)\lambda - 4(x^2+y^2+xy) , $$ hence we have one positive and one negative eigenvalues, surely it is not positive semi-definite.

The bordered Hessian $H$ has determinant (I don't know what your expression in the question is, but it sure doesn't look right): $$ \det (H) = 6 x^4 y + 12 x^3 y^2 + 12 x^2 y^3 + 6 x y^4 >0 \quad \forall x,y>0. $$

Another way to do this is considering the level set

$$F_a := \{(x,y): f(x,y)\ge t\}$$ is convex or not:

as you can tell by the level curve the level set is indeed convex, hence $f$ is quasi-concave. To see it mathematically: set $$ x^2y + xy^2 -t =0 \implies y = -\frac{x}{2} + \sqrt{\frac{x^2}{4} + \frac{t}{x}}, $$ and $$ y'' = \frac{6 t (t+x^3)}{x^{5/2} (4 t+x^3)^{3/2}} >0 \quad \forall t>0, $$ therefore $F_a$ is convex and the quasi-concavity of $f$ follows.