Convergence of GANs The original paper, Generative Adversarial Nets (Goodfellow & al., $2014$), states in section $4.2$:

The subderivatives of a supremum of convex functions include the derivative of the function at the point where the maximum is attained. 

I am not able to understand this statement. Can someone please point me to a mathematical proof or representation behind this concept?
 A: Let 
$f(x)=\sup_{\alpha} f_{\alpha}(x)$
where the functions $f_{\alpha}$ are convex on some convex domain $D$.  It's a standard theorem that $f$ is convex on $D$.  
Suppose that at a particular point $x$ in $D$, 
$\beta= \arg \sup_{\alpha} f_{\alpha}(x)$.
Then
$f(x)=f_{\beta}(x)$.
Let $g$ be any subgradient of $f_{\beta}(x)$.  That is, $g \in \partial f_{\beta}(x)$.  By the definition of the subdifferential, 
$f_{\beta}(y) \geq f_{\beta}(x) + g^{T}(y-x)$ 
for all $y$ in $D$.   
Since $f(y) \geq f_{\beta}(y)$ for all $y$ in $D$, 
$f(y) \geq f(x) + g^{T}(y-x)$
for all $y$ in $D$.  Thus $g \in \partial f(x)$.  
Since this holds for any subgradient $g$ in $\partial f_{\beta}(x)$, 
$\partial f_{\beta}(x) \subseteq \partial f(x)$.  
Note that the authors of this paper have used somewhat inconsistent notation- they write that $\partial f_{\beta}(x) \in \partial f(x)$, but 
in fact the subdifferential of $f_{\beta}$ is a subset of the subdifferential of $f(x)$.  
A: Here is a geometric view of Brian's answer:
Let $\psi(x) = \sup_\alpha f_\alpha(x)$ and suppose $x$ and $\beta$ are such
that $\psi(x) = f_\beta(x)$.
Since $\operatorname{epi} \psi = \cap_\alpha \operatorname{epi}f_\alpha \subset \operatorname{epi}f_\beta$, we
see that any supporting hyperplane to $\operatorname{epi}f_\beta$ at
$x$ (that is, that contains the epigraph 'above' and passes through
$(x,f_\beta(x))$ must also be a supporting hyperplane to
$\operatorname{epi} \psi$.
In particular, $\partial f_\beta(x) \subset \partial \psi (x)$.
To obtain some intuition, look at $\psi(x) = \max(x,-x)$ at
the point $x=0$.
