# Showing that a general Hessian matrix is positive semidefinite

Given vector $$a \in \mathbb{R}^n$$, show that the scalar field $$g : \mathbb{R}^n \to \mathbb{R}$$ defined by $$g(\mathbf{x}) = -\log ( f(\mathbf{x}))$$ where $$f(\mathbf{x}) = \dfrac{1}{1+\exp(-a^T\mathbf{x})}$$ is convex.

To show that, we need to show that the Hessian matrix is positive semidefinite, i.e., $$\nabla \nabla g(\mathbf{x})\succcurlyeq 0$$. I calculated the Hessian as follows:

$$\begin{equation*} \mathbf{H} = \nabla \nabla g(\textbf{x}) = \left[\dfrac{\partial^2 f(\mathbf{x})}{\partial x_i \partial x_j} \right] = \begin{cases} a_i^2 f^2(\mathbf{x})\exp{(\mathbf{-a}^T\mathbf{x})} & , \ \ \text{if} \ \ \ i = j \\ a_ia_j f^2(\mathbf{x})\exp{(\mathbf{-a}^T\mathbf{x})} & , \ \ \text{if} \ \ \ i \neq j \end{cases} \end{equation*}$$

I am not sure what is the easiest way to show the positive semidefiniteness of such a functional form of the Hessian. I cannot see how we could show $$z^T\mathbf{H}z \geq 0$$ for all nonzero $$z$$ or that the eigenvalues of $$\mathbf{H}$$ are all non-negative. Any suggestions?

• You can collect the term $f^2(x)e^{-a^Tx}$, which is nonnegative. So you only need to study the matrix $(a_ia_j)$. You can write it as $aa^T$. Now study the associated quadratic form Jan 30, 2021 at 0:23
• That's very helpful! What do you mean exactly by the "associated quadratic form"? Jan 30, 2021 at 0:35
• I mean the quadratic form $x\in \mathbb R^n \mapsto x^T a a^T x$. It is quite obvious that it is positive semidefinite, because it equals $(a^Tx)^2$. Jan 30, 2021 at 12:41

You do not need the Hessian. The scalar function $$h(s)=\log (1+e^s)$$ is convex since its second derivative is non-negative. Your function is the composition of $$h$$ with the linear function $$m(\mathbf{x})=-a^T\cdot \mathbf{x}$$ so it is convex as well (compositions of convex and linear functions are convex).
• you do not need the second derivative for $h$, it is just log-sum-exp evaluated in $(0,s)$ Jan 30, 2021 at 0:58