8
$\begingroup$

If $X$ is a random variable and $\rho$ is a parameter, and $L$ is a concave function of $(\rho,X)$, under what conditions is the following statement true? $$\mathbb{E}\max_{\rho} L(\rho,X) =\max_{\rho}\mathbb{E}( L(\rho,X)).$$

I can show that $$\mathbb{E}\max_{\rho} L(\rho,X) \geq \max_{\rho}\mathbb{E}( L(\rho,X)),$$ for all $L$ and $X$ (using a proof very similar to that of the min-max theorem). However, I am not able to derive conditions under which the other inequality holds.

$\endgroup$
  • $\begingroup$ Just to be sure: You mean concave in the two variables (i.e. every secant segment lies below the graph of the function of two variables)? $\endgroup$ – Michael Hardy Sep 9 '14 at 0:50
  • $\begingroup$ @MichaelHardy : Yes. Jointly concave in the tuple $(\rho,X)$. $\endgroup$ – Devil Sep 9 '14 at 0:51
11
$\begingroup$

I'm going to use sup rather than max, because in some situations the maximum is not actually attained, and I don't want to bother worrying about that.

It's always true (no concavity necessary) that for any $\rho$ and any $x$, $\sup_\rho L(\rho,x) \ge L(\rho,x)$ so since expected value preserves $\ge$, ${\mathbb E} \sup_\rho L(\rho,X) \ge {\mathbb E} L(\rho, X)$. Taking the supremum of the right side, ${\mathbb E} \sup_\rho L(\rho,X) \ge \sup_\rho {\mathbb E} L(\rho, X)$.

Now, let's turn the question around and ask how could we ensure strict inequality? Suppose there are sets $A$, $B$ with $\mathbb P(X \in A) > 0$ and $\mathbb P(X \in B) > 0$, disjoint sets $C$ and $D$ of $\rho$ values, and $\epsilon > 0$ such that

  1. For $x \in A$, $\sup_{\rho \in C} L(\rho,x) \ge \epsilon + \sup_{\rho \notin C} L(\rho,x)$.
  2. For $x \in B$, $\sup_{\rho \in D} L(\rho,x) \ge \epsilon + \sup_{\rho \notin D} L(\rho,x)$.

For any $x$ and $\rho$, let $Q(\rho,x) = \sup_\rho L(\rho,x) - L(\rho,x) \ge 0$. If $\rho \notin C$, $Q(\rho,x) \ge \epsilon$ when $x \in A$. Thus $\mathbb E \sup_\rho L(\rho, X) - L(\rho,X) \ge \mathbb P(X \in A) \epsilon > 0$. Similarly, if $\rho \notin D$, $Q(\rho,x) \ge \epsilon$ when $x \in B$, and $\mathbb E \sup_\rho L(\rho, X) - L(\rho,X) \ge \mathbb P(X \in B) \epsilon > 0$. But since $C$ and $D$ are disjoint, that covers every $\rho$.

Thus the only way to have equality is that such $A,B,C,D,\epsilon$ do not exist. Essentially, what this means is that those $\rho$ that make $L(\rho,x)$ large for one $x$ must also make it large for all other $x$. For example, if $L(\cdot,x)$ has a unique maximum at some $\rho_x$, then $\rho_X$ must be almost surely constant.

$\endgroup$
  • $\begingroup$ Why use the subscript in $\mathbb E_X$? It doesn't mean anything. $\endgroup$ – Michael Hardy Sep 9 '14 at 1:11
  • $\begingroup$ It was there in the original question. But I'll remove it. $\endgroup$ – Robert Israel Sep 9 '14 at 1:15
  • $\begingroup$ Could you please explain why max may not be attained but sup will be? I thought, it was the other way round...Surely, you can always pick up the max value in a set... Or am I mixing different things here? $\endgroup$ – Lola Jun 1 '17 at 9:54
  • $\begingroup$ Also, shouldn't the last paragraph read "Thus the only way to have INequality" since this is what you are looking for? $\endgroup$ – Lola Jun 1 '17 at 11:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.