0
$\begingroup$

Let $f\in \mathcal C^2(\mathbb R^n)$ that has a local minimum at $a$. I want to prove that the Hessian matrix is positive semi-definite at $a$, i.e. that $$x^TH(f)(a)x\geq 0$$ for all $x$. Since $f\in \mathcal C^2$, if $h\in \mathbb R^n$, there is $|\xi_h|\leq |h|$ s.t. $$0\leq f(a+h)-f(a)=h^TH(f)(\xi_h)h.$$

How can I conclude from here that $$x^TH(f)(a)x\geq 0$$ for all $x\in \mathbb R^n$ ?

$\endgroup$
1
  • 1
    $\begingroup$ Suppose $x \in \mathbb R^n$. Let $g:\mathbb R \to \mathbb R$ be the function defined by $g(t) = f(a + t x)$. Note that $0$ is a minimizer for $g$. From single variable calculus, we know that $g''(0) \geq 0$. But, $g''(0) = x^T Hf(a) x$. So, $x^T Hf(a) x \geq 0$ for all $x \in \mathbb R^n$. This shows that $Hf(a)$ is positive semidefinite. $\endgroup$
    – littleO
    Commented May 18, 2020 at 13:36

1 Answer 1

1
$\begingroup$

You can conclude that $x^{T}H(f)(a)x \ge 0$ for small $x$. Which shows the result for any $x \in \mathbb{R}^{n}$ by definition of positive semidefinite.

EDIT : Let $\bar{x}$ be the local minimum we have $$ f(\bar{x} + h) = f(\bar{x}) + \frac{1}{2} h^{T} H_{f} (\bar{x})h + o(\|h\|^{2}) $$ Becuse $\nabla f (\bar{x}) = 0$.

And finally we have $$ 0 \le f(\bar{x} + h) - f(\bar{x}) \approx_{\| h \| \rightarrow 0} \frac{1}{2} h^{T} H_{f} (\bar{x})h $$ EDIT3 : If $h^{T} H_{f} (\bar{x})h \ge 0$ for $\| h \| \le \delta$ then $h^{T} H_{f} (\bar{x})h$ for all $h \in \mathbb{R}^{n}$.

EDIT2 : As I said it proves semidefinite because definite is false. For exemple the hessian of $f(x,y)= xy$ is $\begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}$ which is only semidefinite.

$\endgroup$
4
  • $\begingroup$ why this show the result for all $x$ if it works for small x ? $\endgroup$
    – Walace
    Commented May 18, 2020 at 13:06
  • $\begingroup$ Because $ f(a+h)-f(a)=h^TH(f)(\xi_h)h + o(\|h\|^{2})$. Let $\bar{x}$ be the local minimum we have $$ f(\bar{x} + h) = f(\bar{x}) + \frac{1}{2} h^{T} H_{f} (\bar{x})h + o(\|h\|^{2}) $$ Becuse $\nabla f (\bar{x}) = 0$. And finally we have $$ 0 \le f(\bar{x} + h) = f(\bar{x}) \approx_{\| h \| \rightarrow 0} \frac{1}{2} h^{T} H_{f} (\bar{x})h $$ $\endgroup$
    – CechMS
    Commented May 18, 2020 at 13:07
  • $\begingroup$ I don't get why this prove that $x^THf(a)x\geq 0$ for all $x\in\mathbb R^n$. $\endgroup$
    – Walace
    Commented May 18, 2020 at 13:17
  • $\begingroup$ Because if $h^{T} H_{f} (\bar{x})h \ge 0$ for $\| h \| \le \delta$ then $h^{T} H_{f} (\bar{x})h$ for all $h \in \mathbb{R}^{n}$. $\endgroup$
    – CechMS
    Commented May 18, 2020 at 13:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .