# Bounding "Jensen's Gap": Elementary Approaches

The point of this post is to explore some "elementary" but general ways one can quantify the "gap" in Jensen's inequality. Specifically, let $$h:\mathbb R\rightarrow\mathbb R$$ be a convex function and let $$X$$ be a real-valued random variable, then how can we bound $$?\leq \mathbb E\big[h(X)\big]-h\big(\mathbb EX\big)\leq ?$$ There does exist a good amount of research on this question, which i will detail in Section 2 of this post, but first i want to give a simple example to explain what i mean by (a) elementary and (b) a "non-trivial bound" on Jensen's gap:

# Section 1: Bounded Second Derivatives

Suppose $$h$$ is twice differentiable and there exists a $$\lambda>0$$ such that $$h''>\lambda$$. Then it is easily seen by computing second derivatives that the function $$g(x)=h(x)-\frac12 \lambda x^2$$ is convex. Thus, applying Jensen's inequality to $$g$$ yields $$\mathbb E\bigg[h(X)-\frac12\lambda X^2\bigg]\geq h\big(\mathbb EX\big)-\frac12 \lambda (\mathbb E X)^2$$ and rearranging gives $$\mathbb E\big[ h(X)\big]-h\big(\mathbb E X\big)\geq \frac12\lambda\bigg[\mathbb EX^2-(\mathbb EX)^2\bigg]=\frac12 \lambda \text{Var}(X).$$ Thus we have achieved a lower bound on Jensen's gap. Similarly, if we assume $$h''<\Lambda$$ then the same argument shows the upper bound $$\mathbb E\big[ h(X)\big]-h\big(\mathbb E X\big)\leq \frac12\Lambda \text{Var}(X).$$ (As a small remark, we technically have only used the assumption that $$h-\frac12\lambda x^2$$ is convex. This can be true even if $$h$$ is not twice differentiable)

I hope you would agree that such a result is "elementary" in the sense that the condition on $$h$$ is both intuitive and likely easy to verify. The proof also is easily understood. The bound this achieves is meaningful in the sense that it uses some score of "how strictly convex" the function $$h$$ is (quantified by $$\lambda$$) and uses a simple property of the distribution of $$X$$. Clearly these two types of information are the least we will need to obtain an interesting bound.

I am essentially looking for arguments and results of similar simplicity that give you a bound on Jensen's gap.

# Section 2: Some existing Research

Before i present some research papers, let me first refer to a few existing StackExchange posts about Jensen's gap: First, there is a question about the gap for $$h(x)=\frac{1}{1+x}$$ and $$X\geq 0$$, which again finds a meaningful upper bound on the gap based on the first two moments of $$X$$. There's a post about the minimal Eigenvalue of a Random Matrix, which again represents a concave function, where no good bound on the gap was found. Also worth noting is this post about $$h(x)=|x|$$.

Now to the interesting part: Here are two research papers i have found on this question:

First, there is "Bound on the Jensen Gap, and Implications for Mean-Concentrated Distributions". Among other things, it first mentions very elementary upper and lower bound the gap for $$\alpha$$-Hölder-continuous functions $$h$$ based on the $$\alpha$$-th moment $$\mathbb E|X-\mathbb EX|^\alpha$$. Some nice examples where e.g. $$h(x)=\log x$$ or $$h(x)=\sqrt x$$ are also presented. Their main results (Theorem 2.1 and Theorem 3.1) seem to generalize the bound based on Hölder-continuity to some functions "locally Hölder-continuous around the mean of $$X$$"

As another example, there is "Some new estimates of the ‘Jensen gap’". Their main results seem to be based on a condition on the Taylor series of the convex function $$h$$. Though the result they obtain in Theorem 1 seems to be of a similar type to the result obtained in Section 1.

Final Remarks: I want to reiterate that i am looking for elementary appraoches to this question, i.e. results where the conditions on $$h$$ and $$X$$ are easy to verify and ideally results with a concise proof. Section 2 was only there to have a "reference post" here on StackExchange for results on Jensen's gap.

• An interesting exercise would be to find an "elementary" result where $h$ has a continuous (or Lipschitz) third, fourth, etc. derivative. Also, just to add to section 2: Lee et al. 2021, "Further sharpening of Jensen's inequality", and Streeter and Dillon, "Automatically Bounding the Taylor Remainder Series: Tighter Bounds and New Applications". Commented Jun 21, 2023 at 2:14
• Just to note: I also have a StackExchange question on a problem related to the Jensen gap. Commented Jun 21, 2023 at 2:16

There are two additional bounds of this kind I am aware of.

Pascu et al (2017). If $$X$$ is a discrete random variable on a closed interval and $$h$$ is continuous, then:

$$|\mathbb{E}[h(X)] - h(\mathbb{E}[X])|\le \omega(h,\eta)\cdot(1+\text{Var}(X)/\eta^2),$$

whenever $$\eta\gt 0$$, where $$\omega(h, \eta)$$ is the modulus of continuity of $$h$$.

Strukov and Timan (1977). If $$X$$ is a discrete random variable on the real line and $$h$$ is continuous, then:

$$|\mathbb{E}[h(X)] - h(\mathbb{E}[X])|\le 3\omega_2(h, (\text{Var}(X))^{1/2}/2),$$

where $$\omega_2(h, \eta)$$ is the second-order modulus of continuity of $$h$$. If both $$X$$ and $$h$$ are supported on a closed interval, the 3 can be replaced with 15.

For both bounds, $$h$$ is not necessarily concave.

References:

• Great answer! Though one remark: You mention that "3 can be replaced by 15", which is clearly true, but in the paper they mention that you can replace $3\omega_2^R$ by $15\omega_2^f$, which is probably what you meant to say. Commented Jun 21, 2023 at 9:23
• @smalldeviation : Edited. Commented Jun 21, 2023 at 9:33

This may not be headed in the direction you want, but in the case where the domain of $$h$$ is an interval, you can get some results that only involve the function values at specific points, without involving derivatives: http://elib.mi.sanu.ac.rs/files/journals/publ/105/n099p107.pdf

(At a glance, this only bounds the gap in one direction, though.)

• This is the kind of result i was looking for, thanks! It's no problem that it's "only in one direction", i'm just looking for any interesting bounds, whether lower or upper bounds. Though I won't accept your answer (yet), since my question was pretty open ended and i want to hear some other responses. Commented Jun 21, 2023 at 8:19
• No problem, hope other interesting answers show up! Commented Jun 21, 2023 at 15:52
• In case the link in this answer breaks, here is another one. Also, the paper is titled On a new converse of Jensen's inequality and was published in Publications de l'Institut Mathematique by Simić Slavko in 2009. Commented Mar 24 at 14:22