21
$\begingroup$

The log-sum-exp function $f: \; \mathbb R^n \to \mathbb R$ is defined by

$$f(x)=\ln \left (e^{x_1}+\cdots + e^{x_n} \right).$$ It is well-known that this function is convex, but I wonder that whether or not $f$ is strictly convex?

Thank you for any answer.

$\endgroup$
1
  • 1
    $\begingroup$ what did you try did you get something for $n=2$ $\endgroup$
    – Elaqqad
    Mar 14, 2015 at 16:49

1 Answer 1

20
$\begingroup$

No, it is not. Consider the case when $x_1 =\dots=x_n=y$. Then $f(x)=y+\log n$. The function is affine along this line, so it is not strictly convex.

In fact, it is affine along any line of the form $x = y \vec{1} + b$, where $b$ is constant: $f(x) = y + \log\sum_i e^{b_i}$.

EDIT: Richkent wants to know if there are any other lines along which $f$ is affine. The answer is no. To see why, let's look at the Hessian of $f(x)$, which has a special structure: $$\nabla f(x) = g, \quad \nabla^2 f(x) = \mathop{\textrm{diag}}(g) - gg^T, \quad g \triangleq \frac{1}{\sum_i e^{x_i}} \begin{bmatrix} e^{x_1} \\ \vdots \\ e^{x_n} \end{bmatrix}$$ Notice that $g\succ 0$ and $\vec{1}^Tg = 1$, so $\vec{1}^T \nabla^2f(x) \vec{1} = 0$. This confirms that $f$ is affine along the directions $v=\alpha\vec{1}$.

But note also that $\nabla^2 f$ is a rank-1 modification of a positive definite matrix $\mathop{\textrm{diag}}(g)$. This is important, because it tells us that $\mathop{\textrm{rank}}(\nabla^2 f(x))=n-1$, and therefore that $v=\alpha\vec{1}$ are the only vectors for which $v^T\nabla^2 f(x) v=0$. Thus $f$ is strictly convex along any other directions.

To see this a bit better, let's look at the function $g(t)=f(x+tv)$, where $(x,v)$ are fixed. This is just a slice of $f$ along the line $x+tv$, so of course it is convex. The second derivative of $g$ is $$g''(t) = v^T \nabla^2 f(x+tv) v.$$ If $v=\alpha\vec{1}$, then $$g''(t) = v^T\nabla^2 f(x+tv) v=\alpha^2\vec{1}^T\nabla^2(x+tv)\vec{1}=0$$ confirming that $g$ is affine if $v=\alpha\vec{1}$. But if $v\neq\alpha\vec{1}$, then $$g''(t) = v^T\nabla^2 f(x+tv) v>0$$ which means that $g$ is strictly convex.

$\endgroup$
3
  • $\begingroup$ I have a further queston that needs your help: Can we determine all the line segment $[x, y]=\{tx+(1-t)y\;:\; 0<t<1 \}$ on which $f$ is affine? $\endgroup$
    – Richkent
    Mar 15, 2015 at 16:10
  • $\begingroup$ There are no other lines. See my edit... $\endgroup$ Mar 15, 2015 at 16:20
  • 1
    $\begingroup$ It is strictly convex nowhere. Its restriction to any line, as long as that line does not have that one direction, is strictly convex. But that is of course a function on $\mathbb{R}$. $\endgroup$ May 8, 2017 at 23:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .