When does it help to write a function as $f(x) = \sup_\alpha \phi_{\alpha}(x)$ (an upper envelope)? In this question I was shown a very elegant solution based on writing a function as the upper envelope of a family of linear functions:
$$f(x) = \sup_{y\in C} f(y) + \langle  \nabla f(y), x-y  \rangle$$
and thereby we showed that $f$ is convex. I wonder: in what other circumstances is it profitable to look at the upper (lower) envelope of a bunch of functions with some property?
One simple result I can think of: $\vec{x} \mapsto \max_i x_i$ is convex because it is the $\sup$ of $n$ linear functions. 
 A: Suppose that we can write a function $f: \mathbb{R} \to \mathbb{R}$ as the upper envelope of a family of functions $(\phi_{\alpha})_{\alpha \in A}$, i.e.
$$f(x) = \sup_{\alpha \in A} \phi_\alpha(x), \qquad x \in \mathbb{R}. \tag{1}$$
Then a lot of properties of the functions $\phi_{\alpha}$ carry over to $f$; for example


*

*lower semicontinuity

*convexity

*subadditivity

*monotonicity

*measurablility (if the index set $A$ is countable)


A special case of $(1)$ is the so-called Legendre transform of a function $g: \mathbb{R} \to \mathbb{R}$ defined as
$$g^\ast(x) := \sup_{\alpha \in \mathbb{R}} \bigg( \alpha \cdot x - g(\alpha) \bigg), \qquad x \in \mathbb{R}$$
One can show that any convex lower semicontinuous function $g$ can be regained by applying this transform twice, i.e.
$$g(\alpha) = \sup_{x \in \mathbb{R}} \bigg( \alpha \cdot x - g^\ast(x) \bigg) = (g^\ast)^{\ast}(\alpha), \qquad \alpha \in \mathbb{R}$$
If $g$ is differentiable, this formula becomes
$$g(\alpha) = \sup_{x \in \mathbb{R}} \bigg( g(x)- g'(x) \cdot (\alpha-x) \bigg)$$
so we are back at the result you mentioned at the beginning. 
The Legendre transform has many applications in large deviation theory (for example Cramér's theorem) as well as in physics (Hamiltonian/Lagrangian formulation in classical mechanics).
Note that these results hold in a more general framework, namely, for functions $g: M \to (-\infty,\infty]$ where $M$ is a (Hausdorff) topological vector space.
A: You can prove Jensen's inequality very quickly. For example, we want to show that for all convex $f$,
$$f\left(\int_Eg(x)d\mu\right)\leq \int_E f(g(x))d\mu$$
Whenever $\mu$ is a probability measure with $\mu(E)=1$. You first trivially show that the inequality is true for all linear functions $f(x)=\phi(x)=ax+b$ (it is in fact an equality in this case). Then you use that wonderful fact that you can write a measurable convex function as a  supremum of affine functions $\phi_\alpha$ and the fact that $\phi_\alpha\leq f$, $f=\sup_\alpha\phi_n$:
\begin{align}
f\left(\int_Eg(x)d\mu\right)&=\sup_\alpha\phi_\alpha\left(\int_Eg(x)d\mu\right)\\
&= \sup \int_E \phi_\alpha(g(x))d\mu\\
&\leq \sup_\alpha \int_E f(g(x))d\mu\\
&=\int_E f(g(x))d\mu
\end{align}
