# Exchanging max and expectation

If $X$ is a random variable and $\rho$ is a parameter, and $L$ is a concave function of $(\rho,X)$, under what conditions is the following statement true? $$\mathbb{E}\max_{\rho} L(\rho,X) =\max_{\rho}\mathbb{E}( L(\rho,X)).$$

I can show that $$\mathbb{E}\max_{\rho} L(\rho,X) \geq \max_{\rho}\mathbb{E}( L(\rho,X)),$$ for all $L$ and $X$ (using a proof very similar to that of the min-max theorem). However, I am not able to derive conditions under which the other inequality holds.

• Just to be sure: You mean concave in the two variables (i.e. every secant segment lies below the graph of the function of two variables)? – Michael Hardy Sep 9 '14 at 0:50
• @MichaelHardy : Yes. Jointly concave in the tuple $(\rho,X)$. – Devil Sep 9 '14 at 0:51

I'm going to use sup rather than max, because in some situations the maximum is not actually attained, and I don't want to bother worrying about that.

It's always true (no concavity necessary) that for any $\rho$ and any $x$, $\sup_\rho L(\rho,x) \ge L(\rho,x)$ so since expected value preserves $\ge$, ${\mathbb E} \sup_\rho L(\rho,X) \ge {\mathbb E} L(\rho, X)$. Taking the supremum of the right side, ${\mathbb E} \sup_\rho L(\rho,X) \ge \sup_\rho {\mathbb E} L(\rho, X)$.

Now, let's turn the question around and ask how could we ensure strict inequality? Suppose there are sets $A$, $B$ with $\mathbb P(X \in A) > 0$ and $\mathbb P(X \in B) > 0$, disjoint sets $C$ and $D$ of $\rho$ values, and $\epsilon > 0$ such that

1. For $x \in A$, $\sup_{\rho \in C} L(\rho,x) \ge \epsilon + \sup_{\rho \notin C} L(\rho,x)$.
2. For $x \in B$, $\sup_{\rho \in D} L(\rho,x) \ge \epsilon + \sup_{\rho \notin D} L(\rho,x)$.

For any $x$ and $\rho$, let $Q(\rho,x) = \sup_\rho L(\rho,x) - L(\rho,x) \ge 0$. If $\rho \notin C$, $Q(\rho,x) \ge \epsilon$ when $x \in A$. Thus $\mathbb E \sup_\rho L(\rho, X) - L(\rho,X) \ge \mathbb P(X \in A) \epsilon > 0$. Similarly, if $\rho \notin D$, $Q(\rho,x) \ge \epsilon$ when $x \in B$, and $\mathbb E \sup_\rho L(\rho, X) - L(\rho,X) \ge \mathbb P(X \in B) \epsilon > 0$. But since $C$ and $D$ are disjoint, that covers every $\rho$.

Thus the only way to have equality is that such $A,B,C,D,\epsilon$ do not exist. Essentially, what this means is that those $\rho$ that make $L(\rho,x)$ large for one $x$ must also make it large for all other $x$. For example, if $L(\cdot,x)$ has a unique maximum at some $\rho_x$, then $\rho_X$ must be almost surely constant.

• Why use the subscript in $\mathbb E_X$? It doesn't mean anything. – Michael Hardy Sep 9 '14 at 1:11
• It was there in the original question. But I'll remove it. – Robert Israel Sep 9 '14 at 1:15
• Could you please explain why max may not be attained but sup will be? I thought, it was the other way round...Surely, you can always pick up the max value in a set... Or am I mixing different things here? – Lola Jun 1 '17 at 9:54
• Also, shouldn't the last paragraph read "Thus the only way to have INequality" since this is what you are looking for? – Lola Jun 1 '17 at 11:08