I'm working on a homework assignment for my artificial intelligence course and I became a bit stuck on one part.

The problem asks to retrieve the optima of the function and then to tell for each optima whether it is a minimum or a maximum.

The function is as follows: $$ f(x,y,z) = x \ln x + y \ln y + z \ln z + \alpha(x + y + z - 1) $$

The gradient was calculated to be: $$ \nabla f(x,y,z) = (\ln x + \alpha + 1)\vec{i} + (\ln y + \alpha + 1)\vec{j} + (\ln z + \alpha + 1)\vec{k} $$

In order to find the critical points I then set $\nabla f(x,y,z) = 0 $, which gave me the following critical points:

$$ (0,0,0),(\frac{1}{e^{\alpha + 1}},\frac{1}{e^{\alpha + 1}},\frac{1}{e^{\alpha + 1}}),(1,1,1) $$

What is the best approach to test if each critical point it is a maximum or a minimum in the function?

I also have calculated the Hessian matrix for an earlier part of this problem. The hessian matrix came to be:

$$ \left[ \begin{array}{ccc} \frac{1}{x} & 0 & 0 \\ 0 & \frac{1}{y} & 0 \\ 0 & 0 & \frac{1}{z} \end{array} \right] $$

Is there a way to utilize the hessian matrix to test for convexity at those points and therefore if convex it is a minimum? The problem with that thought was that when I calculated it out they all turned out to be minimums which doesn't seem right to me.

  • $\begingroup$ Try graphing the function. Use Wolfram Alpha, for example. A picture is worth ... $\endgroup$ – bubba Feb 2 '14 at 2:27
  • $\begingroup$ You've missed 24 stationary points. $\endgroup$ – Jeff Snider Feb 2 '14 at 3:55
  • $\begingroup$ Can you explain how you came to 24? $\endgroup$ – dmcqu314 Feb 2 '14 at 3:58
  • $\begingroup$ Minima are when the Hessian is Positive Definite. Maxima = Negative Definite. Very easy to check for a diagonal matrix. $\endgroup$ – Jeff Snider Feb 2 '14 at 3:59
  • $\begingroup$ @bubba this is a 4d equation it didn't graph too nicely. Or do you mean to graph the gradient vector? $\endgroup$ – dmcqu314 Feb 2 '14 at 4:01

This is tricky around 0. Here is how I would approach it:

The Jacobian is $$\nabla f(x,y,z)=(\ln x+\alpha +1)i+(\ln y+\alpha +1)j+(\ln z+\alpha +1)k.$$

First note that $x,y,z=0$ is not in the domain, but this function has a value in the limit: $$\lim_{x\to0^+} (x\ln x+\alpha x)=0.$$ We treat points where some or all of $x,y,z=0$ as though they are in the domain, because to exclude them means there is no optimum near $(0,0,0)$ although the function is increasing as $x,y,z\to 0^+$ and bounded above in the local region. This means that $(0,0,0)$ is a local maximum: any point with $x,y,z<0$ does not exist, and any nearby point with $x,y,z>0$ takes a smaller value.

We have stationary points for all $x,y,z\in\{0,e^{-\alpha-1}\}$. This means we have $2^3=8$ stationary points to consider.

The Hessian is $$\left(\begin{array}{ccc}\frac{1}{x}&0&0\\0&\frac{1}{y}&0\\0&0&\frac{1}{z}\end{array}\right).$$

We have determined we should treat $(0,0,0)$ as a local maximizer. The Hessian is undefined here, but at a local maximizer it must be negative definite. For a coordinate with a zero value (e.g., $x=0$) we treat that row of the Hessian as though it contains a negative value on the diagonal and is zero elsewhere.

Next consider $(e^{-\alpha-1},e^{-\alpha-1},e^{-\alpha-1})$. At this point the Hessian has positive values along the diagonal and is zero elsewhere: it is positive definite. Thus this point is a local minimizer.

Consider the remaining stationary points, where at least one coordinate is $0$ and at least one coordinate is $e^{-\alpha-1}$. Here the Hessian has a positive value in diagonal in the row corresponding to $e^{-\alpha-1}$ and an effectively negative value in the row corresponding to $0$. Hence these are all saddle points.

Since the function is unbounded above as $x,y,z\to\infty$ there is no global maximizer. But the function is bounded below, hence $(e^{-\alpha-1},e^{-\alpha-1},e^{-\alpha-1})$ is the global minimizer.

  • $\begingroup$ Ah that makes a lot more sense now. Thanks! $\endgroup$ – dmcqu314 Feb 2 '14 at 18:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.