The log-sum-exp function $f: \; \mathbb R^n \to \mathbb R$ is defined by

$$f(x)=\ln \left (e^{x_1}+\cdots + e^{x_n} \right).$$ It is well-known that this function is convex, but I wonder that whether or not $f$ is strictly convex?

Thank you for any answer.

  • $\begingroup$ why there is two sum symbols $\sum$ and $\cdots$? $\endgroup$ – Elaqqad Mar 14 '15 at 16:42
  • $\begingroup$ Sorry, I have just fixed that mistake! $\endgroup$ – Richkent Mar 14 '15 at 16:46
  • $\begingroup$ what did you try did you get something for $n=2$ $\endgroup$ – Elaqqad Mar 14 '15 at 16:49

No, it is not. Consider the case when $x_1 =\dots=x_n=y$. Then $f(x)=y+\log n$. The function is affine along this line, so it is not strictly convex.

In fact, it is affine along any line of the form $x = y \vec{1} + b$, where $b$ is constant: $f(x) = y + \log\sum_i e^{b_i}$.

EDIT: Richkent wants to know if there are any other lines along which $f$ is affine. The answer is no. To see why, let's look at the Hessian of $f(x)$, which has a special structure: $$\nabla f(x) = g, \quad \nabla^2 f(x) = \mathop{\textrm{diag}}(g) - gg^T, \quad g \triangleq \frac{1}{\sum_i e^{x_i}} \begin{bmatrix} e^{x_1} \\ \vdots \\ e^{x_n} \end{bmatrix}$$ Notice that $g\succ 0$ and $\vec{1}^Tg = 1$, so $\vec{1}^T \nabla^2f(x) \vec{1} = 0$. This confirms that $f$ is affine along the directions $v=\alpha\vec{1}$.

But note also that $\nabla^2 f$ is a rank-1 modification of a positive definite matrix $\mathop{\textrm{diag}}(g)$. This is important, because it tells us that $\mathop{\textrm{rank}}(\nabla^2 f(x))=n-1$, and therefore that $v=\alpha\vec{1}$ are the only vectors for which $v^T\nabla^2 f(x) v=0$. Thus $f$ is strictly convex along any other directions.

To see this a bit better, let's look at the function $g(t)=f(x+tv)$, where $(x,v)$ are fixed. This is just a slice of $f$ along the line $x+tv$, so of course it is convex. The second derivative of $g$ is $$g''(t) = v^T \nabla^2 f(x+tv) v.$$ If $v=\alpha\vec{1}$, then $$g''(t) = v^T\nabla^2 f(x+tv) v=\alpha^2\vec{1}^T\nabla^2(x+tv)\vec{1}=0$$ confirming that $g$ is affine if $v=\alpha\vec{1}$. But if $v\neq\alpha\vec{1}$, then $$g''(t) = v^T\nabla^2 f(x+tv) v>0$$ which means that $g$ is strictly convex.

  • $\begingroup$ @ Michael Grant: I don't think that your statement is true: "In fact, it is affine along any line intersecting the origin". In the case $n=2$ on the line $\{(x_1, x_2):\; \;x_2=3x_1\}$ we have $f(x)=\ln(e^{x_1}+e^{3x_1})$. It's not affine at all. $\endgroup$ – Richkent Mar 15 '15 at 6:48
  • $\begingroup$ Agreed. Will modify that. That's what I get for entering the answer into my iphone, away from my desk, where it is easy to verify things! :-) $\endgroup$ – Michael Grant Mar 15 '15 at 13:05
  • $\begingroup$ I have a further queston that needs your help: Can we determine all the line segment $[x, y]=\{tx+(1-t)y\;:\; 0<t<1 \}$ on which $f$ is affine? $\endgroup$ – Richkent Mar 15 '15 at 16:10
  • $\begingroup$ There are no other lines. See my edit... $\endgroup$ – Michael Grant Mar 15 '15 at 16:20
  • 1
    $\begingroup$ It is strictly convex nowhere. Its restriction to any line, as long as that line does not have that one direction, is strictly convex. But that is of course a function on $\mathbb{R}$. $\endgroup$ – Michael Grant May 8 '17 at 23:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.