# Geometric Meaning of D

While we try to find the min/max/saddlepoint, we calculate D

$D=f_{(xx)}*f_{(yy)}-f_{(xy)}^2$

What exactly is the geometric Interpretation of D?

Thank You

-
Have you had a linear algebra course? – Jason DeVito Oct 31 '12 at 12:23
Did you mean $D=f_{xx}f_{yy}-(f_{xy})^2=\dfrac{{\partial}^2{f}}{{\partial}x^2}\cdot \dfrac{{\partial}^2{f}}{{\partial}y^2}-\left(\dfrac{{\partial}^2{f}}{{\partial}x {\partial}y}\right)^2$? If so, it is determinant of Hessian matrix. – M. Strochyk Oct 31 '12 at 12:25
@Jason, No I have had none. – 007resu Oct 31 '12 at 12:49

$D$ is the determinant of the Hessian matrix. Its sign is coordinate invariant at a point where the gradient is zero. Let $\phi:(z,w)\mapsto (x,y)$ be a change of variables, and let $d\Phi$ be the Jacobian matrix. Then we have that the Hessain relative to the $(z,w)$ coordinates _at a point where $f_x = f_y = 0$ is given by

$$\begin{pmatrix} f_{zz} & f_{zw} \\ f_{wz} & f_{ww} \end{pmatrix} = d\Phi^{T} \begin{pmatrix} f_{xx} & f_{xy} \\ f_{yx} & f_{yy} \end{pmatrix} d\Phi$$

So if we take the determinant we have that

$$D^{(z,w)} = |d\Phi|^2 D^{(x,y)}$$

so that the determinant of the Hessian matrix at a critical point always has the same sign relative to any coordinate system.

Now, at a critical point, the first derivative of the function is 0. So to examine the infinitesimal behaviour of the function, we need to look at its second derivatives. (This is the same as in the single-variable case.) Expanding the Taylor expansion to quadratic order we have that

$$f(x,y) \approx f(x_0,y_0) + \frac12 \begin{pmatrix} x - x_0& y - y_0\end{pmatrix} \begin{pmatrix} f_{xx}(x_0,y_0) & f_{xy}(x_0,y_0) \\ f_{yx}(x_0,y_0) & f_{yy}(x_0,y_0)\end{pmatrix} \begin{pmatrix} x-x_0 \\ y-y_0\end{pmatrix}$$

The eigen-vectors of the Hessian matrix (being symmetric, it is diagonalisable) represents the directions in which the graph of $f$ is most and least curved (using the signed curvature). The eigen-values are the curvatures. The local behaviour of our function $f$ is entirely captured by the signs of the eigen-values of the Hessian matrix (in other words, the signature of the Hessian matrix as a symmetric bilinear form):

1. If all the eigenvalues are strictly positive (negative) the point is a strict local minimum (maximum) of the function.
2. If some of the eigenvalues are positive and some of the eigenvalues are negative, then the point is a saddle point.
3. In the degenerate case that some eigenvalues vanish but all the others have the same sign, we may not have enough information to tell whether the points is a local min/max or a saddle point. (Consider the second-order Taylor polynomial for $z = x^2 \pm y^4$.)

Basically, by a local change of variables we can bring the Taylor expansion relative to some coordinates $(z,w)$ as

$$f(z,w) \approx f(0,0) + \lambda_1 z^2 + \lambda_2 w^2$$

The signs of $\lambda_{1,2}$ determine what shape the function looks like near the origin.

-
And, of course, for $2\times 2$ matrices, there is a quick test as to whether or not all the eigenvalues have the same sign - check if the determinant is positive! – Jason DeVito Oct 31 '12 at 12:44
Well, I have had no linear algebra course. But I will take time to try to understand what all this means. Thank You. :) – 007resu Oct 31 '12 at 12:56

The number $D$ is the determinant of the Hessian matrix of $f$, the matrix of the second partial derivatives $$\mathrm{Hess}_f(x,y) = \left( \begin{array}{cc} \frac{\partial^2 f}{\partial x^2} & \frac{\partial^2 f}{\partial x \partial y} \\ \frac{\partial^2 f}{\partial y \partial x} & \frac{\partial^2 f}{\partial y^2} \end{array} \right) = \left( \begin{array}{cc} f_{xx} & f_{xy} \\ f_{yx} & f_{yy} \end{array} \right).$$

Consider first the case of one-variable functions. If $x_0$ was a critical point for $f$, you know that that if $f''(x_0) > 0$, you have a minimum point, if $f''(x_0) < 0$ then you have a maximum point, and if $f''(x_0) = 0$, you don't have enough information and you need to consider higher derivatives. What replaces the number $f(x_0)$ in the two-dimensional case is the matrix $Hess_f(x_0,y_0)$ of all second-order derivatives. You have 4 of those, two being typically equal ($f_{xy} = f_{yx}$), and so $Hess_f(x,y)$ is a symmetric matrix.

The best way to understand the meaning of $D$ is to consider a few simple examples.

1. Take $f(x,y) = x^2 + 2y^2$. Then, $(0,0)$ is a critical point and at that point, the Hessian matrix is given by $$Hess_f(0,0) = \left( \begin{array}{cc} 1 & 0 \\ 0 & 2 \end{array} \right).$$ The determinant is $D = 2 > 0$ and the point $(0,0)$ is clearly a minimum point of $f$.
2. Take $f(x,y) = x^2 - y^2$. Then, $(0,0)$ is a critical point and at that point, the Hessian matrix is given by $$Hess_f(0,0) = \left( \begin{array}{cc} 1 & 0 \\ 0 & -1 \end{array} \right).$$ The determinant is $D = -1 > 0$ and the point $(0,0)$ is clearly a saddle point of $f$. The function $f$ increases along the $x$ axis and decreases along the $y$ axis.
3. Take $f(x,y) = -x^2 - 2y^2$. Then, $(0,0)$ is a critical point and at that point, the Hessian matrix is given by $$Hess_f(0,0) = \left( \begin{array}{cc} -1 & 0 \\ 0 & -2 \end{array} \right).$$ The determinant is $D = 2 > 0$ and the point $(0,0)$ is clearly a minimum point of $f$.

Note that in all our examples, the matrix $Hess_f(x,y)$ was diagonal. The numbers on the diagonal of a matrix are the eigenvalues of that matrix. The determinant is the product of the eigenvalues. If both numbers are positive, you see that the critical point is a minimum point. If both numbers are negative, you see that the critical point is a maximum point. If one number is negative and one is positive, the determinant is negative, you have a saddle point. If one of the numbers is zero, the determinant is zero and you might have a minimum or a maximum or a saddle of some sort (see for example the Monkey Saddle).

So the determinant $D$ tells you whether you have a min/max point ($D > 0$), a saddle point ($D < 0$) or that you don't have enough information ($D = 0$).

Moving from the specific examples, in general, using a change of variables, you can assume that the critical point $(x_0, y_0)$ is $(0,0)$ and that the matrix $Hess_f(x_0,y_0)$ is diagonal. Approximating $f$ with a second order Taylor polynomial reduces the problem to one of the examples we have dealt with above. This can also be generalized to functions of $n$ variables.

-