Understanding proof of the Jensen inequality

Question

I am trying to understand this specific proof of the Jensen inequality.

Let f be a convex function, X a random variable then $f(E(X))\leq E(f(X))$.

Proof: Consider that for all $\omega \in \Omega$:

(*)$f(E(X))+f'(E(X))(X(\omega)-E(X)) \leq f(X(\omega))$

Applying the expected value to this inequality yields the result.

My main Problem is that I do not understand where we get the inequality $f(E(X))+f'(E(X))(X(\omega)-E(X)) \leq f(X(\omega))$.

The rest of the proof seems to be just a plain calculation. Applying the expected value to this inequality gets us $E(f(E(X)))+E(f'(E(X))(X(\omega)-E(X)))\leq E(f(X(\omega)))$.

By the linearity of the expected value and since $f(E(X)),f'(E(X)),E(X)$ are scalars I get

$f(E(X))+f'(E(X))(E(X)-E(X))\leq E(f(X))$.

Thus yielding $f(E(X))\leq E(f(X))$.

I would be happy if someone could tell/explain the origin of the inequality (*).

Answer 1

Denote $a=E(X)$ and $b=X(\omega)$.

Define the function $g(s)=f(a+s(b-a))$

Then by the definition of derivative

$$ g'(0) = \lim_{t\to 0} \frac{g(t)-g(0)}{t} \\ = \lim_{t\to 0} \frac{f(a+t(b-a))-f(a)}{t} \\ = \lim_{t\to 0} \frac{f(tb + (1-t)a)-f(a)}{t} \\ \leq_{\text{convexity}} \lim_{t\to 0} \frac{tf(b) + (1-t)f(a)-f(a)}{t} \\ = f(b)-f(a). $$

On the other hand by the chain rule

$$ g'(0) = f'(a)(b-a) $$

Combining these yields the result.

Answer 2

Assuming $X$ has finite expectation $\mu$, the convexity of $f$ implies that there is at least one real number $b$ such that $f(x)\ge f(\mu)+b(x-\mu)$ for all real $x$. Now let $x=X(\omega)$ and the take expectations.

(Note. Because $f(X)\ge f(\mu)+b(X-\mu)$, the negative part $[f(X)]^-:=\max(0,-f(X))$ of $f(X)$ has finite expectation. Thus, even though the positive part $[f(X)]^+:=\max(0,f(X))$ might have infinite expectation, the expectation of $f(X) = [f(X)]^+-[f(X)]^-$ is well defined, with value in $(-\infty,+\infty]$.)