26
$\begingroup$

Suppose that $X$ is an arbitrary random variable, is the following is true for any function $f$: $$\underset{y\in \mathcal Y} \sup \mathbb E\big[f(X,y)\big] \le \mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]?$$

If $f$ is convex in $X$, then the inequality clearly holds, since the supremum of a family of convex functions is still convex. If $f$ is not convex in $X$, I think the inequality still holds for the following reason:

For any realization of $X$ and any value of $y$, we have $f(X,y) \le \underset{y\in \mathcal Y} \sup f(X,y)$. Therefore, for any $y$, $\mathbb E\big[f(X,y)\big] \le \mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]$. In other words, $\mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]$ is an upper bound of the set $\left\{\mathbb E\big[f(X,y)\big]: y\in \mathcal Y\right\}$, so it follows that $\underset{y\in \mathcal Y} \sup \mathbb E\big[f(X,y)\big] \le \mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]$.

So it appears that convexity of $f$is not needed at all for the inequality to hold. Am I mistaken somewhere? I'd appreciate it if someone would correct me, if I missed something. Thanks a lot!

$\endgroup$
5
  • $\begingroup$ do you mind giving some insights on why for convex functions it is direct please? $\endgroup$
    – Marine Galantin
    Commented Aug 23, 2021 at 11:27
  • 1
    $\begingroup$ @MarineGalantin it's been 4 years...honestly I don't quite recall why I said that then...now I don't see how convexity can be used to prove it either...can it? will consider removing the sentences... $\endgroup$
    – syeh_106
    Commented Aug 25, 2021 at 1:27
  • $\begingroup$ To answer @MarineGalantin's question for posterity: the convex case follows from the argument above via Jensen's inequality. $\endgroup$
    – Danica
    Commented Jan 31, 2022 at 8:05
  • $\begingroup$ @Danica would you elaborate a bit more? I tried to recall...not sure if i said that erroneously at that time...e.g. suppose $f$ is convex in $X$, then so is $\underset{y}\sup f(X,y)$...hence $\mathbb E[\underset{y}\sup f(X,y)]\ge \underset{y}\sup f(\mathbb E[X],y)$...but this doesn't imply $\mathbb E[\underset{y}\sup f(X,y)]\ge \underset{y}\sup \mathbb E[f(X,y)]$ though... $\endgroup$
    – syeh_106
    Commented Feb 15, 2022 at 8:23
  • 1
    $\begingroup$ Oh, huh, I guess you're right @syeh_106 – was thinking sloppily. You could use Jensen's inequality here as follows (maybe annoying to formalize for infinite $\mathcal Y$ though): think of the random vector in $\mathbb R^{|\mathcal Y|}$ that stacks up all the $f(X, y)$ for different $y$, and apply the elementwise-max function to that vector. That's a convex function (max of the [linear] component projection functions), so Jensen's gives the desired inequality. But that has nothing to do with convexity of $f$ anyway. $\endgroup$
    – Danica
    Commented Mar 2, 2022 at 7:38

2 Answers 2

Reset to default
14
$\begingroup$

Edit: sorry - I totally missed you already had the proof in the question - it's correct!

We can equivalently think of this as having a function $f_y$ for each $y$. Then what is always the case is that for each $y$ we have $\sup_y f_y(x)\geq f_y(x)$ for each $x$, and taking the expectation over $X$ this gives $$ \mathbb{E}\left[\sup_y f_y(X)\right]\geq \mathbb{E}\left[f_y(X)\right]$$ Now take the sup over the right side to get the inequality we wanted.

$\endgroup$
1
  • $\begingroup$ When do we have equality? i.e. $E[sup f] = sup E[f]$ Does this holds for if $f$ is a nonnegative function? $\endgroup$
    – user144410
    Commented Mar 28, 2019 at 20:00
4
$\begingroup$

How can you assure $\sup_{y\in \mathcal{Y}} f(X,y)$ is measurable? This is the case if the map $y\mapsto f(x,y)$ is continuous

$\endgroup$
3
  • 5
    $\begingroup$ This is a comment, not an answer. $\endgroup$
    – Taroccoesbrocco
    Commented Sep 19, 2018 at 13:45
  • $\begingroup$ Thanks for pointing this out! I didn't consider measurability at all; what I was getting at was primarily the convexity of $f$ for the inequality to hold. I think I should've said "for any $f$ for which $f(X, y)$ is measurable." $\endgroup$
    – syeh_106
    Commented Sep 21, 2018 at 1:17
  • $\begingroup$ @Mike they can instead use the esssup. $\endgroup$
    – ABIM
    Commented Jan 13, 2023 at 17:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .