Showing that a given function is convex.

Question

I am trying to show that the function

$f(x,\vec{y})=\alpha\ln(1+\exp(x+\vec{y}\cdot \vec{z}))+(1-\alpha)\ln(1+\exp(-x-\vec{y}\cdot\vec{z}))$

is a convex function of $(x,\vec{y})$ (where $x\in\mathbb{R}$,$\vec{y}\in\mathbb{R}^n$, $\alpha\in\{0,1\}$, and $\vec{z}$ is a pre-defined vector in $\mathbb{R}^n$).

I know that it's possible to show that a function of multiple variables is convex if its Hessian matrix is positive semi-definite, but that method seems to be very calculation-intensive, so I was wondering if there is a simpler method that I have overlooked.

I know this would be trivial if the natural log function was convex (as the composition of convex functions is convex, as is the sum of convex functions).

Thanks in advance for any help received!

Answer 1

The easiest way to go is something like this:

• Prove that $f_1(x)=\log(1+\exp(x))$ is a convex function of $x$.
• Note that $f_2(x,y)=\alpha f_1(x) + (1-\alpha) f_2(y)$ is a convex function of $x,y$ for fixed $\alpha\in(0,1)$. The product of a convex function and a positive constant is convex, and the sum of convex functions is convex.
• Note that $f(x,\vec{y})=f_2(x+\vec{y}\cdot\vec{z},-x-\vec{y}\cdot\vec{z})$, the composition of a convex outer function and affine inner functions of $(x,\vec{y})$. Such a convex-affine composition is always convex.

This kind of step-based approach is almost always better than a brute force derivative verification, particularly since it can handle non-differentiable cases! The affine composition rule is not as widely appreciated but it's not difficult to prove from first principles.

For more information, consult Chapter 3 of Convex Optimization by Boyd & Vandenberghe.