Suppose that $X$ is an arbitrary random variable, is the following is true for any function $f$: $$\underset{y\in \mathcal Y} \sup \mathbb E\big[f(X,y)\big] \le \mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]?$$

If $f$ is convex in $X$, then the inequality clearly holds, since the supremum of a family of convex functions is still convex. If $f$ is not convex in $X$, I think the inequality still holds for the following reason:

For any realization of $X$ and any value of $y$, we have $f(X,y) \le \underset{y\in \mathcal Y} \sup f(X,y)$. Therefore, for any $y$, $\mathbb E\big[f(X,y)\big] \le \mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]$. In other words, $\mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]$ is an upper bound of the set $\left\{\mathbb E\big[f(X,y)\big]: y\in \mathcal Y\right\}$, so it follows that $\underset{y\in \mathcal Y} \sup \mathbb E\big[f(X,y)\big] \le \mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]$.

So it appears that convexity of $f$is not needed at all for the inequality to hold. Am I mistaken somewhere? I'd appreciate it if someone would correct me, if I missed something. Thanks a lot!

  • $\begingroup$ do you mind giving some insights on why for convex functions it is direct please? $\endgroup$ Aug 23, 2021 at 11:27
  • 1
    $\begingroup$ @MarineGalantin it's been 4 years...honestly I don't quite recall why I said that then...now I don't see how convexity can be used to prove it either...can it? will consider removing the sentences... $\endgroup$
    – syeh_106
    Aug 25, 2021 at 1:27
  • $\begingroup$ To answer @MarineGalantin's question for posterity: the convex case follows from the argument above via Jensen's inequality. $\endgroup$
    – Danica
    Jan 31 at 8:05
  • $\begingroup$ @Danica would you elaborate a bit more? I tried to recall...not sure if i said that erroneously at that time...e.g. suppose $f$ is convex in $X$, then so is $\underset{y}\sup f(X,y)$...hence $\mathbb E[\underset{y}\sup f(X,y)]\ge \underset{y}\sup f(\mathbb E[X],y)$...but this doesn't imply $\mathbb E[\underset{y}\sup f(X,y)]\ge \underset{y}\sup \mathbb E[f(X,y)]$ though... $\endgroup$
    – syeh_106
    Feb 15 at 8:23
  • 1
    $\begingroup$ Oh, huh, I guess you're right @syeh_106 – was thinking sloppily. You could use Jensen's inequality here as follows (maybe annoying to formalize for infinite $\mathcal Y$ though): think of the random vector in $\mathbb R^{|\mathcal Y|}$ that stacks up all the $f(X, y)$ for different $y$, and apply the elementwise-max function to that vector. That's a convex function (max of the [linear] component projection functions), so Jensen's gives the desired inequality. But that has nothing to do with convexity of $f$ anyway. $\endgroup$
    – Danica
    Mar 2 at 7:38

2 Answers 2


Edit: sorry - I totally missed you already had the proof in the question - it's correct!

We can equivalently think of this as having a function $f_y$ for each $y$. Then what is always the case is that for each $y$ we have $\sup_y f_y(x)\geq f_y(x)$ for each $x$, and taking the expectation over $X$ this gives $$ \mathbb{E}\left[\sup_y f_y(X)\right]\geq \mathbb{E}\left[f_y(X)\right]$$ Now take the sup over the right side to get the inequality we wanted.

  • $\begingroup$ When do we have equality? i.e. $E[sup f] = sup E[f]$ Does this holds for if $f$ is a nonnegative function? $\endgroup$
    – user144410
    Mar 28, 2019 at 20:00

How can you assure $\sup_{y\in \mathcal{Y}} f(X,y)$ is measurable? This is the case if the map $y\mapsto f(x,y)$ is continuous

  • 5
    $\begingroup$ This is a comment, not an answer. $\endgroup$ Sep 19, 2018 at 13:45
  • $\begingroup$ Thanks for pointing this out! I didn't consider measurability at all; what I was getting at was primarily the convexity of $f$ for the inequality to hold. I think I should've said "for any $f$ for which $f(X, y)$ is measurable." $\endgroup$
    – syeh_106
    Sep 21, 2018 at 1:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.