23
$\begingroup$

Consider two real-valued functions of $\theta$, $f(\cdot): \Theta \subset\mathbb{R}\rightarrow \mathbb{R}$ and $g(\cdot):\Theta \subset \mathbb{R}\rightarrow \mathbb{R}$.

Is there any relation between

(1) $\sup_{\theta \in \Theta} (f(\theta)+g(\theta))$

and

(2) $\sup_{\theta \in \Theta} f(\theta)+\sup_{\theta \in \Theta} g(\theta)$

?

Could you provide some informal proof or intuition behind your answer?

$\endgroup$
0

4 Answers 4

38
$\begingroup$

See that $f(\theta) \le \sup_{\Theta}f(\theta)$ for all $\theta \in \Theta$ and also $g(\theta) \le \sup_{\Theta}g(\theta)$ for all $\theta \in \Theta$ and thus you'll have $f(\theta)+g(\theta) \le \sup_{\Theta}f(\theta)+\sup_{\Theta}g(\theta)$ for all $\theta \in \Theta$ and it follows that $\sup_{\Theta} \left ( f(\theta)+g(\theta) \right ) \le \sup_{\Theta}f(\theta)+\sup_{\Theta}g(\theta)$

$\endgroup$
2
  • $\begingroup$ Sorry, but I don't get it how you justify the last step. $\endgroup$
    – mdcq
    Oct 20, 2017 at 14:57
  • 8
    $\begingroup$ $f(\theta)+g(\theta)\le \sup_{\theta}f(\Theta)+\sup_{\Theta}g(\theta)$ for all $\theta \in \Theta$. Therefore the maximum value attained by the function $f+g$ over $\Theta$ must be less than or equal to $\sup_{\Theta} f(\theta) + \sup_{\Theta}g(\theta)$. This maximum value is nothing but the supremum of $f+g$ over $\Theta$. Thus $\sup_{\Theta}(f(\theta)+g(\theta)) \le \sup_{\Theta}f(\theta)+\sup_{\Theta}g(\theta)$ $\endgroup$ Nov 14, 2017 at 19:41
9
$\begingroup$

There is a relationship, and here's the intuition. It works best if we assume that $f$ and $g$ attain their supremums on $\Theta$, but you can extend the intuition to the general case. Let $\theta_f^*$ and $\theta_g^*$ be the values in $\Theta$ at which $f$ and $g$ achieve their supremums, respectively: $$ f(\theta_f^*) = \sup_{\theta \in \Theta} f(\theta), \qquad g(\theta_g^*) = \sup_{\theta \in \Theta} g(\theta). $$

If $f$ and $g$ attain their supremums at the same point $\theta_f^* = \theta_g^* =: \theta^*$, then to maximize $f(\theta) + g(\theta)$, we should pick $\theta = \theta^*$; we know this is right because picking any $\theta$ other than $\theta^*$ will decrease both $f$ and $g$, and hence $f+g$. So in this case, $$ \sup_{\theta \in \Theta} [f(\theta) + g(\theta)] = f(\theta^*) + g(\theta^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta). $$

If $f$ and $g$ do not attain their supremums at the same point, that is, $\theta_f^* \neq \theta_g^*$, then again, the best possible value for $(f + g)(\theta)$ that we could hope for is $f(\theta_f^*) + g(\theta_g^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta)$. But we can only evaluate $f+g$ at one particular $\theta$; maybe we should pick $\theta_f^*$ or maybe $\theta_g^*$ or some other $\theta$, but since $\theta_f^* \neq \theta_g^*$, we cannot simultaneously maximize $f$ and $g$ to maximize $f+g$. Hence, in this case, $$ \sup_{\theta \in \Theta} [f(\theta) + g(\theta)] \leq f(\theta_f^*) + f(\theta_g^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta). $$ This is the inequality which holds most generally. @Siddharth Joshi shows how to prove it (without assuming $f$ or $g$ attains its supremum).

$\endgroup$
8
$\begingroup$

For $\alpha = \sup f \ge f(x)$ for all x, and $\beta = \sup g \ge f(x)$ for all x then $\alpha + \beta \ge f(x) + f(y)$ for all x. So $\sup f + \sup g = \alpha + \beta \ge \sup (f + g)$.

That's what others are calling "intuitive". I like to think of it as degree of limitation. We have more degree of freedom for f and g operating independently than together as a fixed sum (f + g). So the sum of the sups is greater (or equal) than the sup of the sums because we simply have more options.

But that's a pretty vague definition and if it isn't clear when I first say it, it'll just be confusing.

To show that equality might not hold, simply imagine f and g "get big" at different places. Imagine $f(x) < 1$ for all $x \ne 1$ but $f(1) = 1$ (example $f(x) = 1 - (x-1)^2$) $\sup f = 1$. Imagine $g(x) < 1$ for all $x \ne 0$ but $f(0) = 1$ (Ex. $g(x) = 1 - x^x$) $\sup g = 1$. But $f + g$ is always significantly less than 2. (In our examples $f + g = 1 - 2(x^2 - x) \le 3/2$)

Then $\sup f + \sup g > \sup (f + g)$. In our example $2 = \sup f + \sup g > \sup (f + g) = 3/2$

Or better yet let $g(x) = -f(x)$ where $f$ is bounded above and below but includes both positive and negative values. $f(x) + g(x) = 0$ so $\sup(f + g) = 0$ but $\sup f > 0$ and $\sup g = - \inf f > 0$.

$\endgroup$
5
$\begingroup$

To see that equality need not hold, consider $f(\theta) = 1 - g(\theta)$ with $g(\theta)$ positive monotone increasing. We clearly have that $$ \sup_\theta (f(\theta) + g(\theta)) = 1$$ while $$ \sup_\theta(f(\theta)) + \sup_\theta(g(\theta)) = 1 + \infty = \infty$$

Notice that the following inequality always hold $$\sup_\theta(f(\theta)+g(\theta)) \leq \sup_\theta(f(\theta)) + \sup_\theta(g(\theta)) $$ Intuitively, equality holds if and only if $f$ and $g$ achieve their maximal value at the same point $\theta$.

$\endgroup$
2
  • $\begingroup$ are they related by some inequalities? $\endgroup$
    – Star
    Dec 7, 2015 at 16:18
  • $\begingroup$ In your example, you need $0\in g(\Theta)$. $\endgroup$
    – 0xbadf00d
    Dec 6, 2019 at 15:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .