# Sum of the supremum and supremum of a sum

Consider two real-valued functions of $\theta$, $f(\cdot): \Theta \subset\mathbb{R}\rightarrow \mathbb{R}$ and $g(\cdot):\Theta \subset \mathbb{R}\rightarrow \mathbb{R}$.

Is there any relation between

(1) $\sup_{\theta \in \Theta} (f(\theta)+g(\theta))$

and

(2) $\sup_{\theta \in \Theta} f(\theta)+\sup_{\theta \in \Theta} g(\theta)$

?

See that $f(\theta) \le \sup_{\Theta}f(\theta)$ for all $\theta \in \Theta$ and also $g(\theta) \le \sup_{\Theta}g(\theta)$ for all $\theta \in \Theta$ and thus you'll have $f(\theta)+g(\theta) \le \sup_{\Theta}f(\theta)+\sup_{\Theta}g(\theta)$ for all $\theta \in \Theta$ and it follows that $\sup_{\Theta} \left ( f(\theta)+g(\theta) \right ) \le \sup_{\Theta}f(\theta)+\sup_{\Theta}g(\theta)$

• Sorry, but I don't get it how you justify the last step.
– mdcq
Oct 20, 2017 at 14:57
• $f(\theta)+g(\theta)\le \sup_{\theta}f(\Theta)+\sup_{\Theta}g(\theta)$ for all $\theta \in \Theta$. Therefore the maximum value attained by the function $f+g$ over $\Theta$ must be less than or equal to $\sup_{\Theta} f(\theta) + \sup_{\Theta}g(\theta)$. This maximum value is nothing but the supremum of $f+g$ over $\Theta$. Thus $\sup_{\Theta}(f(\theta)+g(\theta)) \le \sup_{\Theta}f(\theta)+\sup_{\Theta}g(\theta)$ Nov 14, 2017 at 19:41

There is a relationship, and here's the intuition. It works best if we assume that $$f$$ and $$g$$ attain their supremums on $$\Theta$$, but you can extend the intuition to the general case. Let $$\theta_f^*$$ and $$\theta_g^*$$ be the values in $$\Theta$$ at which $$f$$ and $$g$$ achieve their supremums, respectively: $$f(\theta_f^*) = \sup_{\theta \in \Theta} f(\theta), \qquad g(\theta_g^*) = \sup_{\theta \in \Theta} g(\theta).$$

If $$f$$ and $$g$$ attain their supremums at the same point $$\theta_f^* = \theta_g^* =: \theta^*$$, then to maximize $$f(\theta) + g(\theta)$$, we should pick $$\theta = \theta^*$$; we know this is right because picking any $$\theta$$ other than $$\theta^*$$ will decrease both $$f$$ and $$g$$, and hence $$f+g$$. So in this case, $$\sup_{\theta \in \Theta} [f(\theta) + g(\theta)] = f(\theta^*) + g(\theta^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta).$$

If $$f$$ and $$g$$ do not attain their supremums at the same point, that is, $$\theta_f^* \neq \theta_g^*$$, then again, the best possible value for $$(f + g)(\theta)$$ that we could hope for is $$f(\theta_f^*) + g(\theta_g^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta)$$. But we can only evaluate $$f+g$$ at one particular $$\theta$$; maybe we should pick $$\theta_f^*$$ or maybe $$\theta_g^*$$ or some other $$\theta$$, but since $$\theta_f^* \neq \theta_g^*$$, we cannot simultaneously maximize $$f$$ and $$g$$ to maximize $$f+g$$. Hence, in this case, $$\sup_{\theta \in \Theta} [f(\theta) + g(\theta)] \leq f(\theta_f^*) + f(\theta_g^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta).$$ This is the inequality which holds most generally. @Siddharth Joshi shows how to prove it (without assuming $$f$$ or $$g$$ attains its supremum).

For $\alpha = \sup f \ge f(x)$ for all x, and $\beta = \sup g \ge f(x)$ for all x then $\alpha + \beta \ge f(x) + f(y)$ for all x. So $\sup f + \sup g = \alpha + \beta \ge \sup (f + g)$.

That's what others are calling "intuitive". I like to think of it as degree of limitation. We have more degree of freedom for f and g operating independently than together as a fixed sum (f + g). So the sum of the sups is greater (or equal) than the sup of the sums because we simply have more options.

But that's a pretty vague definition and if it isn't clear when I first say it, it'll just be confusing.

To show that equality might not hold, simply imagine f and g "get big" at different places. Imagine $f(x) < 1$ for all $x \ne 1$ but $f(1) = 1$ (example $f(x) = 1 - (x-1)^2$) $\sup f = 1$. Imagine $g(x) < 1$ for all $x \ne 0$ but $f(0) = 1$ (Ex. $g(x) = 1 - x^x$) $\sup g = 1$. But $f + g$ is always significantly less than 2. (In our examples $f + g = 1 - 2(x^2 - x) \le 3/2$)

Then $\sup f + \sup g > \sup (f + g)$. In our example $2 = \sup f + \sup g > \sup (f + g) = 3/2$

Or better yet let $g(x) = -f(x)$ where $f$ is bounded above and below but includes both positive and negative values. $f(x) + g(x) = 0$ so $\sup(f + g) = 0$ but $\sup f > 0$ and $\sup g = - \inf f > 0$.

To see that equality need not hold, consider $f(\theta) = 1 - g(\theta)$ with $g(\theta)$ positive monotone increasing. We clearly have that $$\sup_\theta (f(\theta) + g(\theta)) = 1$$ while $$\sup_\theta(f(\theta)) + \sup_\theta(g(\theta)) = 1 + \infty = \infty$$

Notice that the following inequality always hold $$\sup_\theta(f(\theta)+g(\theta)) \leq \sup_\theta(f(\theta)) + \sup_\theta(g(\theta))$$ Intuitively, equality holds if and only if $f$ and $g$ achieve their maximal value at the same point $\theta$.

• are they related by some inequalities?
– Star
Dec 7, 2015 at 16:18
• In your example, you need $0\in g(\Theta)$. Dec 6, 2019 at 15:42