Optimisation in two variables - second order conditions I have a problem understanding second-order conditions at a critical point in finding critical points of two-variable functions.
Let's consider $f(x,y)$.
The first-order conditions are $\frac{\partial f}{\partial x}=0$ and $\frac{\partial f}{\partial y}=0$. So the rate of change of $f$ in respect to both $x$ and $y$ is naught at a critical point.
The second-order conditions at a critical point that I have in my book are of the following form:
A point (a,b) is a maximum if $f_{xx} f_{yy} - f_{xy}^2 > 0 $ and $f_{xx} < 0$.
(For all my functions, $f_{xy}=f_{yx}$.)
What I don't understand is why the second-order conditions have to be so complicated. I think if $f_{xx} > 0$ and $f_{yy} > 0$ (both second-order derivatives are positive), we have a minimum. If $f_{xx}<0$ and $f_{yy}<0$, we have a maximum. And if the signs of two second-order derivatives at (a,b) are different it is a saddle point. What am I missing?
 A: For a minimum at $(x_0,y_0)$ it is necessary that $f(x_0,y_0)$ is minimal with respect to the $f$-values at neighbouring points ${\it all\ around}$ $(x_0,y_0)$. The derivatives $f_{xx}$ and $f_{yy}$ contain only information about changes in $f$ for horizontal resp. vertical displacements, so we must definitely look at $f_{xy}$ to make sure that we don't have a saddle point.
An example: The origin is a critical point for the function $f(x,y):=x^2-4xy+y^2$, and we have $f_{xx}(0,0)=f_{yy}(0,0)=2>0$; so you would diagnose a minimum. But if you pass the origin in $45^\circ$-direction, i.e., via $x(t)=y(t)=t\ \ (-1<t<1)$, then you measure the temperature $\phi(t)=f(x(t),y(t))=-2t^2$, and $f$ seems to be maximal at $(0,0)$. In reality we have a saddle point there.
A: The wikipedia article will provide some insight. What you are looking at is actually the determinant of a $2 \times 2$ Hessian matrix.
A: I actually found a very good illustration of how my simple theory breaks down on page 2 of this file. They have very useful graphs there and after I plotted $f(x,y)=x^2+y^2+2xy$ in Wolfram Alpha, I understood how both second derivatives $f_{xx}$ and $f_{yy}$ can be positive at a critical point yet the function has no unique minimum at that point. I think the Alpha plot shows it very well!
A: To elaborate a bit on PEV's answer: you are considering the Hessian matrix of your function
$$H_f= \left( \begin{array}{cc} 
f_{xx} & f_{xy}\\
f_{xy} & f_{yy} \end{array} \right)$$
then you apply a general theorem for maxima and minima for multi-variable functions which says that if $m$ and $M$ are the smallest and greatest eigevalue of the Hessian and $m > 0$, then the point at which you're evaluating the matrix is a maxmum and if $M < 0$ the point is a minimum and if $mM < 0$ the point is neither a maxmimum nor a minumim.
This condition is complicated because you are considering functions of two variables which behave a lot more complicately than functions of one, i.e. they change in an uncountable number of directions (take for an example directional derivatives). The condition for a stationary point of a single-valued function to be a maximum is that $f'' <0$ but then again, we're moving in only one direction, so that's enough. On the other hand (as Christian points out) for multi-variable functions, you need to have information about a lot more directions than just two (which are given by $f_{xx}$ and $f_{yy}$).
 If you think of it this way this theorem is actually a great simplification of all the work one might have to do without it.
