1
$\begingroup$

Suppose $f(\vec{x}) = - \log{\sigma(\vec{x}^T \vec{w})}$ where $\sigma(y) = \frac{1}{1+e^{-y}}$. Show that the function is convex.

I have computed the gradient to be $\nabla_i f(\vec{x}) = - w_i (1+e^{-\vec{x}^T \vec{w}})$

and the Hessian to be $H_{ij}(\vec{x}) = \nabla^2_{ij} f(\vec{x}) = w_iw_j e^{-\vec{x}^T \vec{w}}$

How can I argue that $H_{ij} \geq 0$ since I don't know the components of $\vec{w}$ and presumably it's possible that $w_i>0$ and $w_j<0$ which would cause a problem. Or is there an implicit assumption that $\vec{w}>0$ which seems unlikely?

$\endgroup$
1
  • $\begingroup$ The matrix $v v^T$ is positive semidefinite, so the Hessian is positive semi definite so the function is convex. $\endgroup$
    – copper.hat
    Sep 6, 2019 at 16:24

1 Answer 1

0
$\begingroup$

Hint

$$f(x)=-\log {e^y\over 1+e^y}\Big|_{y=x^Tw}=-x^Tw-\log {1\over 1+e^y}\Big|_{y=x^Tw}$$therefore we only need to show that $-\log {1\over 1+e^y}\Big|_{y=x^Tw}$ is convex. Now use the theorem stating that if $f:A\to B$ and $g:B\to C$ are convex, then $fog:A\to C$ is convex either.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .