# supremum of expectation $\le$ expectation of supremum?

Suppose that $X$ is an arbitrary random variable, is the following is true for any function $f$: $$\underset{y\in \mathcal Y} \sup \mathbb E\big[f(X,y)\big] \le \mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]?$$

If $f$ is convex in $X$, then the inequality clearly holds, since the supremum of a family of convex functions is still convex. If $f$ is not convex in $X$, I think the inequality still holds for the following reason:

For any realization of $X$ and any value of $y$, we have $f(X,y) \le \underset{y\in \mathcal Y} \sup f(X,y)$. Therefore, for any $y$, $\mathbb E\big[f(X,y)\big] \le \mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]$. In other words, $\mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]$ is an upper bound of the set $\left\{\mathbb E\big[f(X,y)\big]: y\in \mathcal Y\right\}$, so it follows that $\underset{y\in \mathcal Y} \sup \mathbb E\big[f(X,y)\big] \le \mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]$.

So it appears that convexity of $f$is not needed at all for the inequality to hold. Am I mistaken somewhere? I'd appreciate it if someone would correct me, if I missed something. Thanks a lot!

• do you mind giving some insights on why for convex functions it is direct please? Aug 23, 2021 at 11:27
• @MarineGalantin it's been 4 years...honestly I don't quite recall why I said that then...now I don't see how convexity can be used to prove it either...can it? will consider removing the sentences... Aug 25, 2021 at 1:27
• To answer @MarineGalantin's question for posterity: the convex case follows from the argument above via Jensen's inequality. Jan 31 at 8:05
• @Danica would you elaborate a bit more? I tried to recall...not sure if i said that erroneously at that time...e.g. suppose $f$ is convex in $X$, then so is $\underset{y}\sup f(X,y)$...hence $\mathbb E[\underset{y}\sup f(X,y)]\ge \underset{y}\sup f(\mathbb E[X],y)$...but this doesn't imply $\mathbb E[\underset{y}\sup f(X,y)]\ge \underset{y}\sup \mathbb E[f(X,y)]$ though... Feb 15 at 8:23
• Oh, huh, I guess you're right @syeh_106 – was thinking sloppily. You could use Jensen's inequality here as follows (maybe annoying to formalize for infinite $\mathcal Y$ though): think of the random vector in $\mathbb R^{|\mathcal Y|}$ that stacks up all the $f(X, y)$ for different $y$, and apply the elementwise-max function to that vector. That's a convex function (max of the [linear] component projection functions), so Jensen's gives the desired inequality. But that has nothing to do with convexity of $f$ anyway. Mar 2 at 7:38

We can equivalently think of this as having a function $f_y$ for each $y$. Then what is always the case is that for each $y$ we have $\sup_y f_y(x)\geq f_y(x)$ for each $x$, and taking the expectation over $X$ this gives $$\mathbb{E}\left[\sup_y f_y(X)\right]\geq \mathbb{E}\left[f_y(X)\right]$$ Now take the sup over the right side to get the inequality we wanted.
• When do we have equality? i.e. $E[sup f] = sup E[f]$ Does this holds for if $f$ is a nonnegative function? Mar 28, 2019 at 20:00
How can you assure $\sup_{y\in \mathcal{Y}} f(X,y)$ is measurable? This is the case if the map $y\mapsto f(x,y)$ is continuous
• Thanks for pointing this out! I didn't consider measurability at all; what I was getting at was primarily the convexity of $f$ for the inequality to hold. I think I should've said "for any $f$ for which $f(X, y)$ is measurable." Sep 21, 2018 at 1:17