What is the support function $h_K(x)\equiv \sup_{z \in K} \langle z, x \rangle$ of a set $K$? I want to ask that, what is a support function intuitively. It is defined as:
$$\sup_{z \in K} \langle z, x \rangle$$ where $z \in K$, $K$ is a nonempty set. In this formulation, $\langle \cdot, \cdot \rangle$ is inner product.
As a function of $x$, what does it mean? Why it can be useful for instance? Thanks.
 A: If you take $K$ to be convex, the support function is, in some sense, a tool for a dual representation of the set as the intersection of half-spaces.
Let's assume that we're in $\mathbb R^n$ for simplicity. A hyperplane can be characterized by a direction $\boldsymbol x\in\mathbb R^n$ and a scalar $b\in\mathbb R$, let's write $H=(\boldsymbol x;b)$ one such hyperplane, the set of points $\boldsymbol z\in\mathbb R^n$ on the hyperplane $H$ are then given by
$$ \langle \boldsymbol z,\boldsymbol x\rangle \quad = \quad b. $$
The set of points $\boldsymbol z$ lying on one side of the hyperplane $H$ can thus always be written as $\langle \boldsymbol z, \boldsymbol x\rangle\le b$ (modulo a change of sign). So considering 
$$\sup_{\boldsymbol z\in K} \langle \boldsymbol z,\boldsymbol x\rangle $$ 
amounts to finding the $b(\boldsymbol x)$ for the direction $\boldsymbol x$ such that set $K$ lies on one side of the hyperplane $(\boldsymbol x,b(\boldsymbol x))$ or equivalently, such that all $z\in K$ verify $\langle \boldsymbol z, \boldsymbol x\rangle \le b(\boldsymbol x)$.
Then $K$ can be understood as the intersection of all the half-spaces thus defined.
It can maybe be useful to look at a basic example: consider the region $K=[0,1]\times [0,1]$. Then let's consider the $x$-direction with the vector $\boldsymbol v=(1,0)^t$, we get
$$ h_K(\boldsymbol v) = \max_{\boldsymbol w\in K} \langle\boldsymbol w,\boldsymbol v \rangle = \max_{w_1\in[0,1]} w_1 = 1   $$
and the hyperplane $(\boldsymbol v,1)$ (i.e, the vertical line $x=1$) is indeed such that $K$ lies on strictly one side of it. Doing the same thing for the direction $-\boldsymbol v$, and the perpendicular directions will bring us the for sides of the region. This is a bit of a trivial example but hopefully it can help somewhat for the intuition.
