Consider two real-valued functions of $\theta$, $f(\cdot): \Theta \subset\mathbb{R}\rightarrow \mathbb{R}$ and $g(\cdot):\Theta \subset \mathbb{R}\rightarrow \mathbb{R}$.

Is there any relation between

(1) $\sup_{\theta \in \Theta} (f(\theta)+g(\theta))$


(2) $\sup_{\theta \in \Theta} f(\theta)+\sup_{\theta \in \Theta} g(\theta)$


Could you provide some informal proof or intuition behind your answer?


See that $f(\theta) \le \sup_{\Theta}f(\theta)$ for all $\theta \in \Theta$ and also $g(\theta) \le \sup_{\Theta}g(\theta)$ for all $\theta \in \Theta$ and thus you'll have $f(\theta)+g(\theta) \le \sup_{\Theta}f(\theta)+\sup_{\Theta}g(\theta)$ for all $\theta \in \Theta$ and it follows that $\sup_{\Theta} \left ( f(\theta)+g(\theta) \right ) \le \sup_{\Theta}f(\theta)+\sup_{\Theta}g(\theta)$

  • $\begingroup$ Sorry, but I don't get it how you justify the last step. $\endgroup$ – philmcole Oct 20 '17 at 14:57
  • 4
    $\begingroup$ $f(\theta)+g(\theta)\le \sup_{\theta}f(\Theta)+\sup_{\Theta}g(\theta)$ for all $\theta \in \Theta$. Therefore the maximum value attained by the function $f+g$ over $\Theta$ must be less than or equal to $\sup_{\Theta} f(\theta) + \sup_{\Theta}g(\theta)$. This maximum value is nothing but the supremum of $f+g$ over $\Theta$. Thus $\sup_{\Theta}(f(\theta)+g(\theta)) \le \sup_{\Theta}f(\theta)+\sup_{\Theta}g(\theta)$ $\endgroup$ – Siddharth Joshi Nov 14 '17 at 19:41

There is a relationship, and here's the intuition. It works best if we assume that $\Theta$ is compact, so that $f$ and $g$ actually attain their supremums on $\Theta$, but you can extend the intuition to the non-compact case. So assuming $\Theta$ is compact, let $\theta_f^*$ and $\theta_g^*$ be the values in $\Theta$ at which $f$ and $g$ achieve their supremums, respectively: $$ f(\theta_f^*) = \sup_{\theta \in \Theta} f(\theta), \qquad g(\theta_g^*) = \sup_{\theta \in \Theta} g(\theta). $$

If $f$ and $g$ attain their supremums at the same point $\theta_f^* = \theta_g^* =: \theta^*$, then to maximize $f(\theta) + g(\theta)$, we should pick $\theta = \theta^*$; we know this is right because picking any $\theta$ other than $\theta^*$ will decrease both $f$ and $g$, and hence $f+g$. So in this case, $$ \sup_{\theta \in \Theta} [f(\theta) + g(\theta)] = f(\theta^*) + g(\theta^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta). $$

If $f$ and $g$ do not attain their supremums at the same point, that is, $\theta_f^* \neq \theta_g^*$, then again, the best possible value for $(f + g)(\theta)$ that we could hope for is $f(\theta_f^*) + g(\theta_g^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta)$. But we can only evaluate $f+g$ at one particular $\theta$; maybe we should pick $\theta_f^*$ or maybe $\theta_g^*$ or some other $\theta$, but since $\theta_f^* \neq \theta_g^*$, we cannot simultaneously maximize $f$ and $g$ to maximize $f+g$. Hence, in this case, $$ \sup_{\theta \in \Theta} [f(\theta) + g(\theta)] \leq f(\theta_f^*) + f(\theta_g^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta). $$ This is the inequality which holds most generally. @Siddharth Joshi shows how to prove it (without assuming compactness of $\Theta$.)


For $\alpha = \sup f \ge f(x)$ for all x, and $\beta = \sup g \ge f(x)$ for all x then $\alpha + \beta \ge f(x) + f(y)$ for all x. So $\sup f + \sup g = \alpha + \beta \ge \sup (f + g)$.

That's what others are calling "intuitive". I like to think of it as degree of limitation. We have more degree of freedom for f and g operating independently than together as a fixed sum (f + g). So the sum of the sups is greater (or equal) than the sup of the sums because we simply have more options.

But that's a pretty vague definition and if it isn't clear when I first say it, it'll just be confusing.

To show that equality might not hold, simply imagine f and g "get big" at different places. Imagine $f(x) < 1$ for all $x \ne 1$ but $f(1) = 1$ (example $f(x) = 1 - (x-1)^2$) $\sup f = 1$. Imagine $g(x) < 1$ for all $x \ne 0$ but $f(0) = 1$ (Ex. $g(x) = 1 - x^x$) $\sup g = 1$. But $f + g$ is always significantly less than 2. (In our examples $f + g = 1 - 2(x^2 - x) \le 3/2$)

Then $\sup f + \sup g > \sup (f + g)$. In our example $2 = \sup f + \sup g > \sup (f + g) = 3/2$

Or better yet let $g(x) = -f(x)$ where $f$ is bounded above and below but includes both positive and negative values. $f(x) + g(x) = 0$ so $\sup(f + g) = 0$ but $\sup f > 0$ and $\sup g = - \inf f > 0$.


To see that equality need not hold, consider $f(\theta) = 1 - g(\theta)$ with $g(\theta)$ positive monotone increasing. We clearly have that $$ \sup_\theta (f(\theta) + g(\theta)) = 1$$ while $$ \sup_\theta(f(\theta)) + \sup_\theta(g(\theta)) = 1 + \infty = \infty$$

Notice that the following inequality always hold $$\sup_\theta(f(\theta)+g(\theta)) \leq \sup_\theta(f(\theta)) + \sup_\theta(g(\theta)) $$ Intuitively, equality holds if and only if $f$ and $g$ achieve their maximal value at the same point $\theta$.

  • $\begingroup$ are they related by some inequalities? $\endgroup$ – TEX Dec 7 '15 at 16:18
  • $\begingroup$ In your example, you need $0\in g(\Theta)$. $\endgroup$ – 0xbadf00d Dec 6 '19 at 15:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.