1
$\begingroup$

Given vector $a \in \mathbb{R}^n$, show that the scalar field $g : \mathbb{R}^n \to \mathbb{R}$ defined by $$g(\mathbf{x}) = -\log ( f(\mathbf{x}))$$ where $$f(\mathbf{x}) = \dfrac{1}{1+\exp(-a^T\mathbf{x})}$$ is convex.


To show that, we need to show that the Hessian matrix is positive semidefinite, i.e., $\nabla \nabla g(\mathbf{x})\succcurlyeq 0$. I calculated the Hessian as follows:

\begin{equation*} \mathbf{H} = \nabla \nabla g(\textbf{x}) = \left[\dfrac{\partial^2 f(\mathbf{x})}{\partial x_i \partial x_j} \right] = \begin{cases} a_i^2 f^2(\mathbf{x})\exp{(\mathbf{-a}^T\mathbf{x})} & , \ \ \text{if} \ \ \ i = j \\ a_ia_j f^2(\mathbf{x})\exp{(\mathbf{-a}^T\mathbf{x})} & , \ \ \text{if} \ \ \ i \neq j \end{cases} \end{equation*}

I am not sure what is the easiest way to show the positive semidefiniteness of such a functional form of the Hessian. I cannot see how we could show $z^T\mathbf{H}z \geq 0$ for all nonzero $z$ or that the eigenvalues of $\mathbf{H}$ are all non-negative. Any suggestions?

$\endgroup$
3
  • 1
    $\begingroup$ You can collect the term $f^2(x)e^{-a^Tx}$, which is nonnegative. So you only need to study the matrix $(a_ia_j)$. You can write it as $aa^T$. Now study the associated quadratic form $\endgroup$ Jan 30, 2021 at 0:23
  • $\begingroup$ That's very helpful! What do you mean exactly by the "associated quadratic form"? $\endgroup$
    – Morcus
    Jan 30, 2021 at 0:35
  • $\begingroup$ I mean the quadratic form $x\in \mathbb R^n \mapsto x^T a a^T x$. It is quite obvious that it is positive semidefinite, because it equals $(a^Tx)^2$. $\endgroup$ Jan 30, 2021 at 12:41

1 Answer 1

5
$\begingroup$

You do not need the Hessian. The scalar function $h(s)=\log (1+e^s)$ is convex since its second derivative is non-negative. Your function is the composition of $h$ with the linear function $m(\mathbf{x})=-a^T\cdot \mathbf{x}$ so it is convex as well (compositions of convex and linear functions are convex).

$\endgroup$
1
  • 1
    $\begingroup$ you do not need the second derivative for $h$, it is just log-sum-exp evaluated in $(0,s)$ $\endgroup$
    – LinAlg
    Jan 30, 2021 at 0:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .