The question I'd like to ask is about the relevance of higher order derivatives in determining the behaviour of a function around a critical point in the multivariate case. The question occurs towards the end and I am going to try to be clear about what I get and what I don't get.

Now in a single variable case, we know from Taylor expansion around the critical point that

$$\Delta y=\frac{1}{2}f''(x_{0})(x-x_{0})^2+R$$

where R is the remainder term which goes to zero one degree faster than the quadratic term, thus if $$f''(x_{0})=0$$, the behaviour around the critical point, i.e. the sign of $$\Delta y$$ is determined by the sign of the third derivative, assuming that its non zero and so on.

In two variable case, we know from Taylor expansion around the critical point that

$$\Delta y=\frac{1}{2}f_{xx}(x_{0},y_{0})(x-x_{0})^2+f_{xy}(x_{0},y_{0})(x-x_{0})(y-y_{0})+\frac{1}{2}f_{yy}(x_{0},y_{0})(y-y_{0})^2+R$$

Now if the determinant of the Hessian matrix is strictly positive, then depending on the sign of $$f_{xx}$$, delta y is always positive or negative so we either have a local minimum or a local maximum. Similarly, if the determinant of the Hessian is strictly negative, then there are some (x,y) where the change is positive, and some (x,y) where the change is negative, so the critical point is a saddle point.

Now what I don't get is, if the determinant is zero, then the polynomial becomes

$$\Delta y=\frac{1}{2}f_{xx}(x_{0},y_{0})((x-x_{0})+\frac{f_{xy}(x_{0},y_{0})}{f_{xx}(x_{0},y_{0})}(y-y_{0}))^2+R$$

(completing the squares) for which the quadratic part is always positive or always negative, assuming that $$f_{xx}(x_{0},y_{0})$$ is non-zero (I don't have any problems with the case when it is zero). So how come the higher-order derivatives are relevant, since always positive or always negative implies local min or local max? The common explanation is that "the derivative is degenerate in one direction so the behaviour in that direction is determined by the higher order terms". I don't get this explanation from the expression above.


Note: I can get the intuition of the examples where partials drop down completely easily, i.e. for the ones where if you expand and take a Taylor expansion around (0,0), all the partials of second and third (or k-1th) order drops, so you are only left with the 4th (or kth) order term, which determines the behaviour of the function around (0,0).

The examples that throw my intuition off are the ones (unfortunately, I can't provide an explicit function), where $$f_{xx}(critical)\neq 0,f_{yy}(critical)\neq 0,f_{xy}(critical)\neq 0 $$, but $$f_{xx}(critical)f_{yy}(critical)-f_{xy}(critical)^2=0$$. In those cases, the second order part of the Taylor expansion drops only partially, not completely. Is that because it loses part of it, it stops dominating the effects of higher order terms?

  • $\begingroup$ The formula "$$\Delta y=\frac{1}{2}f_{xx}(x_{0},y_{0})((x-x_{0})+\frac{f_{xy}(x_{0},y_{0})}{f_{xx}(x_{0},y_{0})}(y-y_{0}))^2+R$$ is wrong. The coefficient of $(y-y_0)^2$ is $(f_{xx}f_{yy}-f_{xy}^2)/f_{xx}$, which vanishes when the discriminant is $0$. $\endgroup$ – Christian Blatter Sep 11 '12 at 14:28

In order to get a feeling of what is possible you should look at polynomial functions $(x,y)\mapsto f(x,y)$ with $f(0,0)=f_x(0,0)=f_y(0,0)=0$. Such functions are their own Taylor expansions to start with, and the character of the citical point $(0,0)$ is immediately visible in the sign of $f(x,y)$ for various $(x,y)$ near $(0,0)$.

The generic case is that $f(x,y)=a x^2 + 2b x y + c y^2 +{\rm higher\ terms}$ with $ac -b^2\ne 0$. In this case you have a local max, a local min or a saddle point at $(0,0)$.

Consider now as an example the function $$f(x,y)=(y-x^2)(y+x^2)\ .$$ Its zero set consists of the two parabolas $y=\pm x^2$, and it is easy to see that $f$ takes positive as well as negative values in the immediate neighborhood of $(0,0)$; whence we don't have a local extremum there. If $f$ were given in a more disguised form (and/or would have additional higher terms) you would have to look at the fourth degree Taylor polynomial of $f$ to find this out through analyzing partial derivatives.

This example tells you that in the case of a degenerate critical point heavy algebraic machinery could be needed for a final diagnosis of this point.

  • $\begingroup$ Thanks for the example, I elaborated my question a bit because those kind of examples are the ones that I am comfortable with. The ones that throws me off are where partials are non-zero but determinant is zero. $\endgroup$ – firemind Mar 25 '12 at 17:58
  • $\begingroup$ @firemind: If $f$ were of the form $$f(x,y):=\bigl(ax+by-(cx +dy)^2\bigr)\bigl(ax+by+(c'x+d'y)^2\bigr)$$ with numerical values for $a$, $\ldots$, $d'$ the second partials would typically be nonzero, but the determinant of the Hessian is zero. $\endgroup$ – Christian Blatter Mar 25 '12 at 18:26
  • $\begingroup$ I'm not gonna ask how you come up with that, but this is a beautiful example. I worked on $$(2 x + 3 y - (6 x + 7 y)^2)*(2 x + 3 y + (6 x + 7 y)^2)$$ a bit, and found out that at the critical point $$(0,0)$$, $$f_{xx}$$ and $$f_{yy}$$ are positive but the determinant is zero. Plotting the function, I saw that the second term (quadratic) in Taylor expansion has actually a critical (global min) at (0,0). $\endgroup$ – firemind Mar 25 '12 at 20:04
  • $\begingroup$ This shows that the infinitesimal changes from x and y does not effect the TOTAL DERIVATIVE, i.e. even though going purely in x or purely in y direction infinitesimally, increases $$f_{x}$$ or $$f_{y}$$ (evidence in favour of local min), we cannot tell anything about directions in which we change both x and y, because the effect of the cross partial cancels out the effect of second partials. Thanks a lot. $\endgroup$ – firemind Mar 25 '12 at 20:08
  • $\begingroup$ @firemind: Any nonzero quadratic polynomial $f(x,y)$ with discriminant $0$ can be written in the form $(ax+by)^2$ with $(a,b)\ne(0,0)$. $\endgroup$ – Christian Blatter Mar 25 '12 at 20:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.