0
$\begingroup$

Consider a convex function $f(x) = x^2$

Let $E_{AV} = \frac{1}{M} \sum_{m=1}^{M} \mathbb{E}_x [(y_m(x) - f(x))^2]$ be average expected sum-of-squares error of the members of an ensemble model

Let $E_{ENS} = \mathbb{E}_x [(\frac{1}{M} \sum_{m=1}^{M} y_m(x) - f(x))^2]$ be the expected error of an ensemble model.

How can we show that $\mathbb{E}_{ENS} \le \mathbb{E}_{AV}$ by applying Jensen Inequality?

$\endgroup$
2
  • $\begingroup$ You have typed the same thing for $E_{AV}$ and $E_{ENS}$ $\endgroup$ Nov 1, 2021 at 7:58
  • $\begingroup$ Now, corrected the typo. $\endgroup$ Nov 1, 2021 at 8:14

1 Answer 1

2
$\begingroup$

$(\frac{1}{M} \sum_{m=1}^{M} (y_m(x) - f(x)))^2 \leq \frac{1}{M} \sum_{m=1}^{M} (y_m(x) - f(x))^2$ by Jensen's Inequality applied to the uniform measure on $\{1,2,...,M\}$. Now just take expectation on both sides.

$\endgroup$
2
  • $\begingroup$ Thank you for the answer. Is the above result holds for any error function, not just sum-of-squares, as long as it is convex? $\endgroup$ Nov 1, 2021 at 8:32
  • $\begingroup$ Yes, the only ingredient of my proof is convexity of $x \to x^{2}$. @TuhinDutta $\endgroup$ Nov 1, 2021 at 8:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .