From convex set to convex function

Question

The definition of a convex set is geometrically intuitive. But the definition of convex function doesn't seem so intuitive: $S \subset \mathbb{R}^n$ is convex if given $x,y\in S$ the line segment joining $x,y$ is in $S$.

Let $f$ be a real valued function from an open interval $I$. Consider the graph of $f$ in plane: set theoretically it is $$\{(x,f(x))\,|\, x\in I\}.$$ For any $a,b\in I$, consider the line joining $(a,f(a))$ and $(b,f(b))$. Then one of the following happens:

1. The line lies above the graph of $f$.

2. The line lies below the graph of $f$.

3. None of these hold.

Q. Suppose you know the definition of convex set and let $f:I\rightarrow \mathbb{R}$ be a function which does not satisfy (3). This means $f$ satisfies (1) or (2). What is intuition way to define function to be convex or concave?

For example, in Wikipedia, it says that a function is convex if graph above $f$ is convex. But if we are trying to give intuitive definition of convex function based on convex set, according to convexity of region above $f$ or below $f$, what is intuitive way to decide one of them?

---

Since $f$ is satisfying (1) or (2), so the words convex function and concave function are reserved for it; we can assign any word to any case without intuition. But, considering definition of convex set can we get intuition to define convex function? Note that almost everyone knows geometric explanation of the standard definition of convex function.

Answer 1

In the article of wikipedia about convex functions the relation between the two concepts is stated in this way:

A function $f:\Bbb R^n\to\Bbb R$ is convex if its epigraph (the set of points on or above the graph of the function) is a convex set.

where the epigraph of a function as the above is defined formally as follow:

$$\operatorname{epi}(f):=\left\{(x,\mu)\in\Bbb R^n\times\Bbb R: \mu\ge f(x)\right\}\tag1$$

Answer 2

I'm refer Boris T. Polyak in his book 'INTRODUCTION TO OPTIMIZATION', page 8.

Definition of convex function

A scalar function $f(x)$ on $\mathbb R^n$ is said to be convex if

$$f(\lambda x+(1-\lambda)y)\leq \lambda f(x) + (1-\lambda)f(y)$$

for any $x,y\in \mathbb R^n$ and $0\leq \lambda \leq 1$.

Theorem

Let $f:\mathbb R^n \to \mathbb R$. $f$ is a convex function if and only if $H_{\gamma}=\{(x,\gamma)|\gamma \geq f(x)\}$ is a convex set.

Now, look this theorem who is so powerful.

Theorem

A function $f$ is convex if and only if $\nabla^2f(x)\geq 0$.

$\nabla^2f(x)$ are Hessian matrix and $\nabla^2f(x)\geq 0$ refer nonnegative definite.

Example

Let $g(x,y) = x^2-xy+y^2$.

$$\implies \nabla g(x,y) = (2x-y,-x+2y)$$ Then \begin{equation} \nabla^2g(x,y) = \left(\begin{smallmatrix} 2 & -1\\ -1 & 2 \end{smallmatrix}\right) \end{equation}

$$\implies \nabla^2g(x,y)\geq 0,\forall x,y\in \mathbb R$$

$$\therefore g(x,y) \text{ is a convex function}$$

Note that if $\nabla^2f(x)$ are not a constant matrix but is nonnegative definite you can not say $f(x)$ are a convex function because this needs to be true on all variables of the function by definition of convex function.