1
$\begingroup$

My textbook said:
To use two-variate calculus to verify that a non-negative function $H(\theta_1,\theta_2)$ has a local maximum at $(\hat\theta_1,\hat\theta_2)$, it must be shown that the following three conditions hold.

  • a. The first-order partial derivatives are $0$, $$\frac{\partial}{\partial\theta_1}H(\theta_1,\theta_2)|_{\theta_1=\hat\theta_1,\theta_2=\hat\theta_2}=0$$ and $$\frac{\partial}{\partial\theta_2}H(\theta_1,\theta_2)|_{\theta_1=\hat\theta_1,\theta_2=\hat\theta_2}=0$$

  • b. At least one second-order partial derivative is negative, $$\frac{\partial^2}{\partial\theta_1^2}H(\theta_1,\theta_2)|_{\theta_1=\hat\theta_1,\theta_2=\hat\theta_2}<0$$ or $$\frac{\partial^2}{\partial\theta_2^2}H(\theta_1,\theta_2)|_{\theta_1=\hat\theta_1,\theta_2=\hat\theta_2}<0$$

  • c. The Jacobian of the second-order partial derivatives is positive, $$\begin{vmatrix} \frac{\partial^2}{\partial\theta_1^2}H(\theta_1,\theta_2)&\frac{\partial^2}{\partial\theta_1\partial\theta_2}H(\theta_1,\theta_2)\\ \frac{\partial^2}{\partial\theta_1\partial\theta_2}H(\theta_1,\theta_2)&\frac{\partial^2}{\partial\theta_2^2}H(\theta_1,\theta_2) \end{vmatrix}_{\theta_1=\hat\theta_1,\theta_2=\hat\theta_2}\\ =\frac{\partial^2}{\partial\theta_1^2}H(\theta_1,\theta_2)\frac{\partial^2}{\partial\theta_2^2}H(\theta_1,\theta_2)-\left(\frac{\partial^2}{\partial\theta_1\partial\theta_2}H(\theta_1,\theta_2)\right)^2 >0$$

I can't understand condition (b): For (b) I think both of the second-order partial derivatives should be negative if only one second-order partial derivative is negative and the other is positive, then it should be a saddle point. And (c) also implies that both of the second-order partial derivatives should be negative.

For (c), since $f(\theta) \approx f(\theta_0) + \nabla f(\theta_0)^T (\theta - \theta_0) + \frac12 (\theta - \theta_0)^T \nabla^2 f(\theta_0) (\theta - \theta_0)$. If $\theta_0$ is a local extremum for $f$, then $\nabla f(\theta_0) = 0$, so if $\nabla^2 f(\theta_0)$ is negative definite, then $(\theta - \theta_0)^T \nabla^2 f(\theta_0) (\theta - \theta_0) \leq 0$ for all $\theta$, so $\theta_0$ is a local maximum. Then the eigenvalues of $\nabla^2 f(\theta_0)$ should be both negative and the determinant should be positive. Am I correct?

$\endgroup$
11
  • 1
    $\begingroup$ It's not possible for only one second-order derivative to be negative while maintaining the determinant of the Jacobian positive. Ie. $(c)$ being true implies that if $(b)$ holds, both second-order derivatives are negative. $\endgroup$ Sep 30, 2021 at 17:20
  • 1
    $\begingroup$ For b) note that negative is $<0$. It could also be $=0$. In fact, the conditions are only sufficient, but not nescessary. Take $H(x,y) = -x^4-y^4$. Then $H$ has it’s only maximum at $(x,y) = (0,0)$, but the Hessian vanishes. Thus the direction you need to prove: If those three properties hold, then $H$ has a local maximum in that point. $\endgroup$
    – Lazy
    Sep 30, 2021 at 17:21
  • $\begingroup$ For this you might want to prove: If b), c) holds then the Hessian of $H$ is strictly negative definite. The proof for this is quite straightforward. $\endgroup$
    – Lazy
    Sep 30, 2021 at 17:33
  • $\begingroup$ I think I already proved $H$ is negative definite: "the eigenvalues of $\nabla^2 f(\theta_0)$ should be both negative", then $H$ is negative definite. $\endgroup$
    – Dan Li
    Sep 30, 2021 at 17:38
  • $\begingroup$ As far as I read you wrote: If it is a local maximum then the hessian is negative definite (not true by the way), then all the eigenvalues are negative, and thus the det positive. So you’d do a circular proof (where even one step is false) if you reasoned from that that the hessian was negative. $\endgroup$
    – Lazy
    Sep 30, 2021 at 17:40

1 Answer 1

1
$\begingroup$

Elaborating on the earlier comments:

For b) note that negative is $<0$. It could also be $=0$. In fact, the conditions are only sufficient, but not nescessary. Take $H(x,y)=−x^4−y^4$. Then $H$ has it’s only maximum at $(x,y)=(0,0)$, but the Hessian vanishes. Thus the direction you need to prove: If those three properties hold, then $H$ has a local maximum in that point.

For this you might want to prove: If b), c) holds then the Hessian of H is strictly negative definite. The proof for this is quite straightforward:

Suppose the Hessian is given by $h_{11},h_{22},h_{12}=h_{21}$. From b) and c) you can easily see the in fact in b) both conditions must hold. Else: Suppose $h_{11}<0,h_{22}\geq 0$ then $h_{11}h_{22} - h_{12}^2 \leq 0$.

Then from c) we get $$ |h_{12}|<\sqrt{h_{11}h_{22}} $$ Then if $v=(v_1,v_2)$, then: $$ v^T\nabla^2 H v = v_1(v_1h_{11}+v_2h_{12})+v_2(v_1h_{12}+v_2h_{22}) = v_1^2 h_{11} + v_2^2h_{22} + 2v_1v_2h_{12} < v_1^2 h_{11} + v_2h_{22} \pm 2v_1v_2\sqrt{h_{11}h_{22}} $$ (where the $<$ holds as long as we do not have $v_1=v_2=0$ and the $\pm$ is so that $\pm v_1v_2\geq0$). Now as $h_{11},h_{22}< 0$ we have $\sqrt{h_{11}}=r_1i$, $\sqrt{h_{22}}=r_2i$, $r_1,r_2>0$. Thus we have for $v\neq 0$ $$ v^T\nabla^2 H v < v_1^2 h_{11} + v_2h_{22} \pm 2v_1v_2\sqrt{h_{11}h_{22}} = (v_1r_1i)^2 + (v_2r_2i)^2 \mp 2(v_1r_1iv_2r_2i) = (v_1r_1i \mp v_2r_2i)^2 \leq 0$$ (with $=$ exactly if $v_1r_1 = \pm v_2r_2$, but that does not matter).

$\endgroup$
2
  • $\begingroup$ Can't $\left(\frac{\partial^2}{\partial\theta_1\partial\theta_2}H(\theta_1,\theta_2)\right)^2 $ quantity in $\frac{\partial^2}{\partial\theta_1^2}H(\theta_1,\theta_2)\frac{\partial^2}{\partial\theta_2^2}H(\theta_1,\theta_2)-\left(\frac{\partial^2}{\partial\theta_1\partial\theta_2}H(\theta_1,\theta_2)\right)^2 >0$ be larger than $\frac{\partial^2}{\partial\theta_1^2}H(\theta_1,\theta_2)\frac{\partial^2}{\partial\theta_2^2}H(\theta_1,\theta_2)$ making the determinant less than $0$? $\endgroup$
    – time
    Dec 29, 2021 at 7:20
  • $\begingroup$ @time As I said, I’m elaborating on some comments. If you look at the second comment to the question I stated that these conditions are sufficient but not nescessary. I even gave a counterexample. $\endgroup$
    – Lazy
    Dec 29, 2021 at 15:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.