# Convexity and equality in Jensen's inequality

The problem:

Recall that we saw that when $\mu$ is a probability measure on $X$ and $f$ is integrable with respect to $\mu$, then $$\exp\left(\int_X f(x)d\mu(x)\right) \leq \int_X e^{f(x)}d\mu(x).$$ What can you conclude if we have equality?

I know the solution is the functions that are constant on $X$ for all but a set of measure 0, but I'm not too sure how to prove this. So far what I've done is to show that a constant function on a set of measure 1 in $X$ gives equality, but I haven't been able to come up with a way to show a function with at least two distinct values on subsets of $X$ of positive measure cannot give equality. Namely:

If we have some integrable function (with respect to $\mu$) that is constant on all but a set of measure 0, say $f(x) = c$ for all $x\in Y\subset X$, with $\mu(Y) = 1$, then the left hand side is \begin{align*} \exp\left(\int_X f(x)d\mu(x)\right) &= \exp\left(c\int_X d\mu(x)\right)\newline &= \exp\left(c\int_Y d\mu(x)\right)\newline &= \exp(c\mu(Y))\newline &= e^c. \end{align*} The right hand side is \begin{align*} \int_X e^{f(x)}d\mu(x) &= \int_X e^c d\mu(x)\newline &= e^c \int_X d\mu(x)\newline &= e^c \int_Y d\mu(x)\newline &= e^c \mu(Y)\newline &= e^c, \end{align*} and so we have $$\exp\left(\int_X f(x)d\mu(x)\right) = \int_X e^{f(x)}d\mu(x).$$ These are the only possible $f$ for equality to occur. To see this, suppose $f$ were not constant (on all but a set of measure 0). Start with the simplest case, namely, suppose $X = N\cup Y\cup Z$ with $N,Y$ and $Z$ pairwise disjoint, $\mu(Y) > 0 < \mu(Z)$, and $\mu(N) = 0$ such that $f(y) = a$ for all $y\in Y$ and $f(z) = b$ for all $z\in Z$, with $a\neq b$.

Then the left hand side is \begin{align*} \exp\left(\int_X f(x)d\mu(x)\right) &= \exp\left(\int_Y a d\mu(x) + \int_Z b d\mu(x)\right)\newline &= \exp\left(a\int_Y d\mu(x) + b\int_Z d\mu(x)\right)\newline &= \exp(a\mu(Y) + b\mu(Z)). \end{align*} whereas the right hand side is \begin{align*} \int_X e^{f(x)}d\mu(x) &= \int_Y e^a d\mu(x) + \int_Z e^b d\mu(x)\newline &= e^a \int_Y d\mu(x) + e^b \int_Z d\mu(x)\newline &= e^a \mu(Y) + e^b \mu(Z). \end{align*}

So...? I'm not sure how to show $a\neq b$ implies $$\exp(a\mu(Y) + b\mu(Z)) < e^a \mu(Y) + e^b \mu(Z).$$

My guess is it's something simple I'm missing, but I don't really see how to proceed.

Edit: I suppose I can add some of the fiddling around I've done.

We know $$\mu(Z) = 1 - \mu(Y),$$ so the original equality can be written as $$\exp(a\mu(Y) + b(1-\mu(Y))) \leq e^a \mu(Y) + e^b (1-\mu(Y))$$ which means $$\exp((a-b)\mu(Y) + b) \leq (e^a - e^b)\mu(Y) + e^b.$$ Hence as $$\exp((a-b)\mu(Y) + b) = \exp((a-b)\mu(Y))e^b,$$ we have $$\exp((a-b)\mu(Y)) \leq (e^{a-b} - 1)\mu(Y) + 1.$$

Edit some more:

I suppose one could look at this last inequality as a function of $a-b$. To make things neater, just a function of $\exp(a-b)$, so let $x = \exp(a-b)$. Then we have $$x^{\mu(Y)} \leq (x - 1)\mu(Y) + 1.$$ Now we know if $a = b$ (i.e., $x = 1$) we have equality, so we can compute the derivative of both sides. The derivative of the left hand side is $$\mu(Y)x^{\mu(Y) - 1},$$ whereas the derivative of the right hand side is simply $\mu(Y)$. Hence as $x$ increases the functions can never be equal again.

I think this works, though I'll sit on it and think a little more about it (and it's generalization to all non-constant functions on $X$)

-

Your argument does not seem to be correct. You wrote:

To see this, suppose $f$ were not constant (on all but a set of measure 0). Namely, suppose $X = N\cup Y\cup Z$ with $N,Y$ and $Z$ pairwise disjoint, $\mu(Y) > 0 < \mu(Z)$, and $\mu(N) = 0$ such that $f(y) = a$ for all $y\in Y$ and $f(z) = b$ for all $z\in Z$, with $a\neq b$.

This would mean that $f$ has only two values, $a$ and $b$. But non-constant functions can have many various values, take just $f(x)=x$.

EDIT: See OP's comments bellow - it seems that I just misunderstood his proof strategy.

Hint 1: Work with the constant $c=\int_X f(x) \,\mathrm{d}\mu(x)$. What can you say if you assume that $\{x;f(x)\ne c\}$ has non-zero measure?

Hint 2: Simply try to go through the proof of Jensen's inequality and on the place where convexity of $e^x$ is used, use the fact that it is strictly convex.

Proof: Suppose that $$\exp\left(\int_X f(x)d\mu(x)\right) = \int_X e^{f(x)}d\mu(x). \qquad (1)$$ We want to show that if equality holds then $f$ is constant.

Let us denote $c=\int_X f(x) \,\mathrm{d}\mu(x)$. The above inequality can be now rewritten as $e^c = \int_X e^{f(x)} \,\mathrm{d}\mu(x)$

Suppose that $f$ is not constant, i.e. $f(x)\ne c$ on a set of positive measure. Then both sets $A=\{x; f(x)>c\}$ and $B=\{x; f(x)<c\}$ must have positive measure. (Otherwise we would get contradiction with the definition of $c$ as the mean value of $f$.)

For any $t\ne c$ we have $e^t>e^c+e^c(t-c)$, since the graph of $e^x$ lies above the tangent line at the point $c$. (The factor $e^c$ is the slope at the point $c$.) This means $$e^t-e^c>e^c(t-c).$$

Thus for any $x\in A\cup B$ we have strict inequality $e^{f(x)}-e^c>(f(x)-c)e^{c}$. For any $x$ we have $e^{f(x)}-e^c \ge (f(x)-c)e^c$. Integrating gives $$\int_X e^{f(x)} \,\mathrm{d}\mu(x) - e^c > e^c \left(\int_X f(x) \,\mathrm{d}\mu(x) -c \right) = 0,$$ contradicting the equality (1).

The same argument works for any strictly convex function instead of $e^x$, see e.g. Lieb, Loss: Analysis, p.45. (Perhaps my answer would be clearer and simpler if I worked with arbitrary strictly convex function. I should have thought about this sooner...)

-
Ah, I was a little unclear. I was planning on proving that we had an inequality if $f$ takes on only two values on all but a set of measure 0, then generalize, though I don't quite see a way for that yet. As for your hint 2, that is a ridiculously clear idea! The proofs of Jensen's inequality with a strictly convex function makes this so simple! I can't believe I didn't think of that. Thanks! – Alex Nov 13 '11 at 6:50
You wrote: I can't believe I didn't think of that. Well, to be honest, for me it was not simple, I spent quite some time on it. (To put it in another words - many proofs are simple if you know the right trick or if you come up with the correct idea. But finding the right trick/basic idea is not that easy.) – Martin Sleziak Nov 13 '11 at 6:58
Heh, true enough. Thanks again :) – Alex Nov 13 '11 at 16:01