# Quadratic surface maximization and Hessians

If we have that the contours of a response surface are elliptical and the response is given by the following function:

$$\large \exp\left(-\left(w^2 + \frac{1}{4}l^2 -\frac{1}{4} \cdot w \cdot l\right)\right)$$

then if we maximize this function w.r.t $l$ holding $w$ fixed at $1/2$.

And if we call the maximizer l-star, then holding l-star fixed, maximize over w. How to show that the overall max isn't achieved?

My approach: I got the partial of the above function w.r.t. $l$ and then tried to evaluate it at $1/2$, but got stuck. It most likely will involve some analysis of Hessians.

-
Welcome to MSE! It really helps readability to format using mathJax (see FAQ). Regards – Amzoti Apr 25 '13 at 1:45
Thank you! I hope you can read it – Richard Wang Apr 25 '13 at 1:49
Yes, it looks good. thank you – Richard Wang Apr 25 '13 at 1:59
I did some editing of the mathematical notation. Writing \exp instead of exp not only prevents italicization but provides for proper spacing in expressions like $a\exp b$. And parentheses assume their proper sizes when you write \left( and \right). And $e^x$ (with a superscript) means the same thing as $\exp x$ (without a superscript). The point of the latter notation is to make the superscript unnecessary, which makes things easier when the expression in the exponent is long or complicated. – Michael Hardy Apr 25 '13 at 4:28
@MichaelHardy, thank you for the edit. How does the following response below answer the question on showing how overall max isn't achieved? Is there any analysis with the Hessian which you can do to make the answer rigorous? Thanks – Richard Wang Apr 27 '13 at 6:31

To maximize $\displaystyle\exp\left(-\left(w^2+\frac14\ell^2-\frac14w\ell\right)\right)$ is to minimize $w^2+\frac14\ell^2+\frac14w\ell$, and that is the same as minimizing $4w^2+\ell^2+w\ell$. As a function of $\ell$, this is $$\ell^2+w\ell+4w^2$$ $$= \left(\ell^2+w\ell + \frac{w^2}{4}\right) - \frac{w^2}{4} + 4w^2\tag{completing the square}$$ $$= \left(\ell^2+w\ell + \frac{w^2}{4}\right) + \frac{15w^2}{4} = \left(\ell+\frac w2\right)^2 + \frac{15w^2}{4}.$$ The value of $\ell$ that minimizes this is $-w/2$, since that makes the square equal to $0$. the whole expression is then $15w^2/4$, and it is easy to find the value of $w$ that minimizes that.

-
could you put in the details of fixing the w=1/2, and the 2* and l*? I ran through a calculation of it, and got l* = -1/16 and w* = -1/8, but want to trace through it to check my work. thanks – Richard Wang Apr 27 '13 at 19:31
I'm not sure where you're getting $w=1/2$. My answer never mentions setting $w$ to $1/2$. Might you have meant my statement that the value of $\ell$ that minimizes a certain expression is $-w/2$? ${}\qquad{}$ – Michael Hardy Apr 28 '13 at 16:16

The exponential function does not actually matter, because it is strictly increasing. The maxima and minima of $\exp(f(w,l))$ are attained (or not attained) precisely at the same points as maxima and minima of $f(w,l)$ itself. One advantage of working with $f(w,l)=-(w^2+l^2/4 - l/2)$ is that its derivatives are simpler than for $e^f$.

And another advantage that we don't even need calculus: it's a quadratic polynomial in which we can complete the square. $$-(w^2 + l^2/4 - l/2) =- (w^2 + (l-1)^2/4-1/4) \tag1$$ To maximize $-(\dots)$, we minimize the content of the parentheses. The smallest $w^2$ can be is $0$, at $w=0$. The smallest $(l-1)^2/4$ can be is $0$, at $l=1$. Therefore, at $w=0$, $l=1$ the global maximum of (1) is attained, and it is equal to $1/4$.

Consequently, $e^f$ has maximum value $e^{1/4}$.

-
But how does this show that the overall max isnt achieved? – Richard Wang Apr 27 '13 at 6:31
In the question as it (now?) appears above, there is a $wl$ term in the quadratic polynomial, which this answer neglects. – Michael Hardy Apr 27 '13 at 15:50
I see: It was edited. – Michael Hardy Apr 27 '13 at 15:56