# Could someone give a detailed (yet elementary) proof for Jensen's inequality?

I want to prove that

Suppose there is a function $f:[a,b] \to \mathbb R$, and there are $x_i \in [a,b], w_i \gt 0$ for $i=1,\dots,n$ such that $\sum_{i=1}^nw_i=1$, then if the function is convex, the following inequality holds $$\sum_{i=1}^nw_if(x_i) \ge f(\sum_{i=1}^n w_ix_i)$$ and if the function is concave, flip the sign of inequality.

Please include (very) detailed explanation, because I do not have much mathematical maturity, and perhaps may not be able to understand what the equation defining convex means if the explanation is too concise.

• First you will need a definition of "convex function". Aug 2 '15 at 13:00
• How about you go through a proof yourself and tell us where you're stuck?
– D_S
Aug 2 '15 at 13:57
• I was told that a function is said convex if Jensen's inequality holds, so if we agree there is very little to prove. We may prove that midpoint convexity plus continuity imply convexity, for instance, but that is just a definition. Aug 2 '15 at 21:27

There is a proof by induction that I read as a note in an open directory of university notes site for a related course, see [1]. I indicate a detailed outline:

I am assuming $\omega_i\geq 0$.

Show that case $n=1$ is trivial. What is the value of $\omega_{1}$ in $f(\omega_1 x_1)\leq \omega_1f(\omega_1)$?

Derive the case $n=2$ directly (trivially) from definition of your convex function. What is the relation between $\omega_1$ and $\omega_2$? Previous answer help you to understand why you can use your definition of convex function (if your definition of a convez function is written in term of $\lambda$ terms, ask you what is the relation between weigths $\omega_1$, $\omega_2$ and these $\lambda$ terms).

Write the statement for $N=k-1$, the so called inductive hypothesis.

Define the following modified weigths $\widehat\omega_i$ satisfying $(1-\omega_k)\cdot\widehat\omega_i=\omega_i$, for each $1\leq i\leq k-1$.

From $\sum_{i=1}^{k-1}\omega_i f(x_i)+\omega_k f(x_k)$ show that it is equal to $(1-\omega_k)\cdot\sum_{i=1}^{k-1}\widehat\omega_i f(x_i)+\omega_k f(x_k)$. Now we can use the inductive hypothesis. But before, recognize the purpose of this manipulation (of terms and weigths versus the condition of lambda terms in definition of a convex function) that we have made?

After the use of inductive hypothesis we obtain

$$\sum_{i=1}^{k-1}\omega_i f(x_i)+\omega_k f(x_k)\geq (1-\omega_k)f\left(\sum_{i=1}^{k-1} \widehat\omega_i x_i\right)+f(x_k)\omega_k$$ and this, by case 2 (basically the definition) is geatest or equal than $$f\left((1-\omega_k)\sum_{i=1}^{k-1} \widehat\omega_i x_i+\omega_k x_k \right)$$

And now go back first two steps, using the definition of new weights and the decomposition of the sum.

Sorry by my english. I hope this helps and there aren't missprints. After your computations, in a days you can read the reference. There are a nice graph too.

References:

[1] Konstantinos Derpanis, Jensen's Inequality (March 12, 2005), page 2.

Lemma: If you have some weighted points scattered in a plane, then their centre of mass lies in their convex hull. To understand convex hull, see the pictures on page-768 of the paper

Proof:

Suppose the point was outside and let's say we marked that in red, then we can draw a line through that point (one in blue), now if we were to draw vectors from that point to vertices where the particles are, it is clear that the vector sum can't be zero because the vector to each particle has some component along the normal. Now, recall the definition of the center of mass,

$$\sum_{i=0}^n m_i \vec{r_i} =(\sum_{i=0}^n m_i)\vec{r_{com} }$$

Or,

$$\sum_{i=0}^n m_i( \vec{r_i} - \frac{\vec{r_{com} } }{n} )= 0$$

So, the centre of mass can be though of the point from which the vectors to the mass particles add to zero, but here it is clear that this doesn't happen. Hence, it is necessary that centre of mass lies inside the convex hull.

I'll do the proof for a convex function $$(f'' \geq 0)$$ and I'll leave it to you to prove it for a concave one. A convex function is something which looks like this:

Now, suppose I placed a mass of $$m$$ on three points on the curve:

It is easy to see the centre of mass of these points must have coordinates $$( \frac{x_1 + x_2 + x_3}{3} , \frac{ f(x_1) + f(x_2) + f(x_3)}{3})$$ (purple dot)

And, we can that the point on curve at same x coordinate (green dot) is given as: $$( \frac{x_1 + x_2 + x_3}{3} , f( \frac{x_1 + x_2 +x_3}{3}))$$

And since the purple dot is above the green dot, we can write the following inequality:

$$f( \frac{x_1 + x_2 +x_3}{3}) \leq \frac{ f(x_1) + f(x_2) + f(x_3)}{3}$$

And of course, this argument would still if you took more points / had different amount of mass on each point. Hence, we have found a simple visual understanding of Jensen's inequality.