I would like to know whether the following function $f: \mathbf{R}^4 \to \mathbf{R}$ is concave or not:

$$ f(x_1, x_2, x_3, x_4) = \log(e^{x_1} + e^{x_2}) - \log(e^{x_1} + e^{x_2} + e^{x_3} + e^{x_4}) $$

I tried to check whether the Hessian was negative semi-definite, but did not get anywhere. The Hessian can be written as

$$ \nabla^2 f(x) = \frac{1}{\tilde{Z}^2}(\tilde{Z} \cdot \text{diag}(\tilde{\mathbf{z}}) - \tilde{\mathbf{z}}\tilde{\mathbf{z}}^\intercal) - \frac{1}{Z^2}(Z \cdot \text{diag}(\mathbf{z}) - \mathbf{z} \mathbf{z}^\intercal), $$

where $$ \tilde{\mathbf{z}} = \begin{bmatrix} e^{x_1} & e^{x_2} & 0 & 0 \end{bmatrix}^\intercal \\ \mathbf{z} = \begin{bmatrix} e^{x_1} & e^{x_2} & e^{x_3} & e^{x_4} \end{bmatrix}^\intercal \\ \tilde{Z} = e^{x_1} + e^{x_2} \\ Z = e^{x_1} + e^{x_2} + e^{x_3} + e^{x_4} $$ but I did not get much further than that. Any help would be greatly appreciated!

This question discusses in detail the convexity of the log-sum-exp function, but does not apply to my case (difference of fcts).

  • 3
    $\begingroup$ Fixing $x_2=x_3=x_4=0$, we get the function $\log(e^x+1)-\log(e^x+3)$. Take the second derivative, simplify, see that it changes sign. $\endgroup$ – user147263 Jun 28 '16 at 11:30
  • 2
    $\begingroup$ @Bookend, thanks, that's simple & clear! Would you like to write a proper answer for it? I'd be glad to accept it. $\endgroup$ – lum Jun 29 '16 at 9:22
  • 13
    $\begingroup$ @Bookend, I asked a question on meta about this: meta.math.stackexchange.com/questions/23554 - would love to get more explanations from you why you think this is a bad question. $\endgroup$ – lum Jun 29 '16 at 12:10
  • 12
    $\begingroup$ @Bookend This question is of perfect use to others. I, for one, found it very informative. $\endgroup$ – The Count Jun 29 '16 at 21:04

(This answer builds up on the first comment to the question above).

Short answer: no, the function is not concave.

Instead of analyzing the full $4 \times 4$ hessian, we can start by restricting our attention to a subspace of the input, e.g., the line induced by setting $x_2 = x_3 = x_4 = 0$. Along this line, the original function can be rewritten as a univariate function $\tilde{f}(x) = \log(e^x + 1) - \log(e^x + 3)$.

the second derivative of $\tilde{f}$ is $$ \frac{d^2\tilde{f}}{dx^2} = \frac{e^x(6 - 2 e^{2x})}{(e^x + 1)^2 (e^x + 3)^2} $$ it is easy to see that the second derivative is positive for $x = 0$, hence $\tilde{f}$ is not concave (furthemore, it is negative for $x = 1$, so it is not convex either). We conclude that $f$ is not concave.

Note that the type of functions described here (difference of log-sum-exps) appears in the log-likelihood function of certain statistical models of paired comparisons, such as Elimination by Aspects and team comparisons.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.