Exchanging max and expectation If $X$ is a random variable and $\rho$ is a parameter, and $L$ is a concave function of $(\rho,X)$, under what conditions is the following statement true?
$$\mathbb{E}\max_{\rho} L(\rho,X) =\max_{\rho}\mathbb{E}( L(\rho,X)).$$
I can show that 
$$\mathbb{E}\max_{\rho} L(\rho,X) \geq \max_{\rho}\mathbb{E}( L(\rho,X)),$$
for all $L$ and $X$ (using a proof very similar to that of the min-max theorem). However, I am not able to derive conditions under which the other inequality holds.
 A: I'm going to use sup rather than max, because in some situations the maximum is not actually attained, and I don't want to bother worrying about that.
It's always true (no concavity necessary) that for any $\rho$ and any $x$,
$\sup_\rho L(\rho,x) \ge L(\rho,x)$ 
so since expected value preserves $\ge$, 
${\mathbb E} \sup_\rho L(\rho,X) \ge {\mathbb E} L(\rho, X)$.  Taking the supremum of the right side, ${\mathbb E} \sup_\rho L(\rho,X) \ge \sup_\rho  {\mathbb E} L(\rho, X)$.
Now, let's turn the question around and ask how could we ensure strict inequality? Suppose there are sets $A$, $B$ with $\mathbb P(X \in A) > 0$ and 
$\mathbb P(X \in B) > 0$,  disjoint sets $C$ and $D$ of $\rho$ values, and $\epsilon > 0$
such that


*

*For $x \in A$, $\sup_{\rho \in C} L(\rho,x) \ge \epsilon + \sup_{\rho \notin C} L(\rho,x)$.

*For $x \in B$, $\sup_{\rho \in D} L(\rho,x) \ge \epsilon + \sup_{\rho \notin D} L(\rho,x)$.


For any $x$ and $\rho$, let $Q(\rho,x) = \sup_\rho L(\rho,x) - L(\rho,x) \ge 0$.
If $\rho \notin C$, $Q(\rho,x) \ge \epsilon$ when $x \in A$.
Thus $\mathbb E \sup_\rho L(\rho, X) - L(\rho,X) \ge \mathbb P(X \in A) \epsilon > 0$.
Similarly, if $\rho \notin D$, $Q(\rho,x) \ge \epsilon$ when $x \in B$,
and $\mathbb E \sup_\rho L(\rho, X) - L(\rho,X) \ge \mathbb P(X \in B) \epsilon > 0$.  But since $C$ and $D$ are disjoint, that covers every $\rho$.
Thus the only way to have equality is that such $A,B,C,D,\epsilon$ do not exist.
Essentially, what this means is that those $\rho$ that make $L(\rho,x)$  large for one $x$ must also make it large for all other $x$.  For example, if 
$L(\cdot,x)$ has a unique maximum at some $\rho_x$, then $\rho_X$ must be 
almost surely constant.
