3
$\begingroup$

To prove that a one dimensional differentiable function $f(x)$ is convex, it is quite obvious to see why we would check whether or not its second derivative is $>0$ or $<0.$

What is the intuition behind the claim that, if the Hessian $H$ of a multidimensional differentiable function $f(x_1,...,x_n)$ is positive semi-definite, it must be convex$?$

Is there a way I interpret this requirement, $y'H(x)y \ge 0\, \forall y$, geometrically$?$ (as in the case of one dimensional $f(x)$) How to interpret vector $y$ here$?$

$\endgroup$
6
$\begingroup$

Recall that a fixed matrix $H\in\mathbb{R}^{n\times n}$ is positive semidefinite if $y^THy\geq 0$ for all $y\in\mathbb{R}^n$, and positive definite if and only if $y^THy>0$ for all nonzero $y$.

Now consider the function $g_{x,y}(t)=f(x+ty)$, where $f$ is a function on $\mathbb{R}^n$ and $x,y$ are fixed vectors. The derivatives at $t=0$ are $$g_{x,y}'(0) = y^T \nabla f(x), \qquad g_{x,y}''(0) = y^T \nabla^2 f(x) y$$ Of course if $g_{x,y}''(0)\geq 0$, then $g_{x,y}$ is convex at the origin. If $g_{x,y}''(0)\geq 0$ for all $y$, then $f$ is convex at $x$. Finally, if $g_{x,y}''(0)\geq 0$ for all $x,y$, then $f$ is convex.

So the intiution is this: a multivariate function is convex if and only if it is convex along every line. The vector $y$ represents the direction of the test line.

$\endgroup$
2
  • $\begingroup$ "Direction of the test line" is good way to think of it, thanks. $\endgroup$ Jan 5 '16 at 11:57
  • 1
    $\begingroup$ Nice answer. For functions which can take the value $+\infty$ at places, if I remember well, as an additional condition you want the effective domain $\text{dom }f$ to be convex. Also, the positive semi-definite assumption can be limited to the interior of $\text{dom }f$. $\endgroup$
    – dohmatob
    Jan 6 '16 at 14:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.