0
$\begingroup$

Consider the logistic loss function

$$\ell(x, y,w) = \log \left( 1 + \exp \left(- y w^T x \right) \right)$$

where $x \in \Bbb R^d$ is an input sample and $y \in \{0,1\}$ is its label. We know that logistic loss is convex w.r.t model parameter $w$, so the Hessian matrix $H = \nabla_{w}^2 \ell(x,y,w)$ is positive semidefinite.

Is $\mbox{trace}[H]$ also a convex function w.r.t $w$? Can we generalize this result for any convex loss function?

$\endgroup$
3
  • $\begingroup$ Isn't $H$ merely a sub matrix of the actual Hessian? $\endgroup$
    – Rodrigo de Azevedo
    Apr 5, 2021 at 13:34
  • $\begingroup$ @RodrigodeAzevedo but $H$ itself is not convex, for $H$ to be convex you require something about the fourth derivative of $\ell$ $\endgroup$
    – LinAlg
    Apr 5, 2021 at 13:39
  • $\begingroup$ please mark the question as answered $\endgroup$
    – LinAlg
    Apr 20, 2021 at 13:30

1 Answer 1

Reset to default
1
$\begingroup$

No, even in the simple case of $d=x=y=1$, $H = e^w / (1+e^w)^2$ is not convex (just plot it).

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .