About the strictly convexity of log-sum-exp function

The log-sum-exp function $f: \; \mathbb R^n \to \mathbb R$ is defined by

$$f(x)=\ln \left (e^{x_1}+\cdots + e^{x_n} \right).$$ It is well-known that this function is convex, but I wonder that whether or not $f$ is strictly convex?

• why there is two sum symbols $\sum$ and $\cdots$? – Elaqqad Mar 14 '15 at 16:42
• Sorry, I have just fixed that mistake! – Richkent Mar 14 '15 at 16:46
• what did you try did you get something for $n=2$ – Elaqqad Mar 14 '15 at 16:49

No, it is not. Consider the case when $x_1 =\dots=x_n=y$. Then $f(x)=y+\log n$. The function is affine along this line, so it is not strictly convex.

In fact, it is affine along any line of the form $x = y \vec{1} + b$, where $b$ is constant: $f(x) = y + \log\sum_i e^{b_i}$.

EDIT: Richkent wants to know if there are any other lines along which $f$ is affine. The answer is no. To see why, let's look at the Hessian of $f(x)$, which has a special structure: $$\nabla f(x) = g, \quad \nabla^2 f(x) = \mathop{\textrm{diag}}(g) - gg^T, \quad g \triangleq \frac{1}{\sum_i e^{x_i}} \begin{bmatrix} e^{x_1} \\ \vdots \\ e^{x_n} \end{bmatrix}$$ Notice that $g\succ 0$ and $\vec{1}^Tg = 1$, so $\vec{1}^T \nabla^2f(x) \vec{1} = 0$. This confirms that $f$ is affine along the directions $v=\alpha\vec{1}$.

But note also that $\nabla^2 f$ is a rank-1 modification of a positive definite matrix $\mathop{\textrm{diag}}(g)$. This is important, because it tells us that $\mathop{\textrm{rank}}(\nabla^2 f(x))=n-1$, and therefore that $v=\alpha\vec{1}$ are the only vectors for which $v^T\nabla^2 f(x) v=0$. Thus $f$ is strictly convex along any other directions.

To see this a bit better, let's look at the function $g(t)=f(x+tv)$, where $(x,v)$ are fixed. This is just a slice of $f$ along the line $x+tv$, so of course it is convex. The second derivative of $g$ is $$g''(t) = v^T \nabla^2 f(x+tv) v.$$ If $v=\alpha\vec{1}$, then $$g''(t) = v^T\nabla^2 f(x+tv) v=\alpha^2\vec{1}^T\nabla^2(x+tv)\vec{1}=0$$ confirming that $g$ is affine if $v=\alpha\vec{1}$. But if $v\neq\alpha\vec{1}$, then $$g''(t) = v^T\nabla^2 f(x+tv) v>0$$ which means that $g$ is strictly convex.

• @ Michael Grant: I don't think that your statement is true: "In fact, it is affine along any line intersecting the origin". In the case $n=2$ on the line $\{(x_1, x_2):\; \;x_2=3x_1\}$ we have $f(x)=\ln(e^{x_1}+e^{3x_1})$. It's not affine at all. – Richkent Mar 15 '15 at 6:48
• Agreed. Will modify that. That's what I get for entering the answer into my iphone, away from my desk, where it is easy to verify things! :-) – Michael Grant Mar 15 '15 at 13:05
• I have a further queston that needs your help: Can we determine all the line segment $[x, y]=\{tx+(1-t)y\;:\; 0<t<1 \}$ on which $f$ is affine? – Richkent Mar 15 '15 at 16:10
• There are no other lines. See my edit... – Michael Grant Mar 15 '15 at 16:20
• It is strictly convex nowhere. Its restriction to any line, as long as that line does not have that one direction, is strictly convex. But that is of course a function on $\mathbb{R}$. – Michael Grant May 8 '17 at 23:11