# Difference between logarithm of an expectation value and expectation value of a logarithm

Assuming I have a always positive random variable $X$, $X \in \mathbb{R}$, $X > 0$. Then I am now interested in the difference between the following two expectation values:

1. $E \left[ \ln X \right]$
2. $\ln E \left[ X \right]$

Is one maybe always a lower/upper bound of the other?

Since the function logarithm is concave, Jensen's inequality shows that $E(\ln(X))\leqslant \ln E(X)$ and that equality occurs iff $X$ is almost surely constant.

Edit (This is to expand on a remark made by Shai.)

Shai's answer explains how to prove $E(\ln(X))\leqslant \ln E(X)$ using only AM-GM inequality and the strong law of large numbers. These very tools yield the following refinement (adapted from the paper Self-improvement of the inequality between arithmetic and geometric means by J. M. Aldaz).

Apply AM-GM inequality to the square roots of an i.i.d. sequence of positive random variables $(X_i)$, that is, $$\sqrt[n]{\sqrt{X_1}\cdots\sqrt{X_n}}\leqslant\frac1n(\sqrt{X_1}+\cdots+\sqrt{X_n}).$$ In the limit $n\to\infty$, the strong law of large numbers yields $$\exp(E(\ln\sqrt{X}))\leqslant E(\sqrt{X}),$$ that is, $$E(\ln X)\leqslant 2\ln E(\sqrt{X})=\ln (E(X)-\mbox{var}(\sqrt{X})).$$ Finally:

For every positive integrable $X$, $$E(\ln X)\leqslant [\ln E(X)]-\delta(X)\quad\mbox{where}\ \delta(X)=\ln[E(X)/E(\sqrt{X})^2].$$

The correction term $\delta(X)$ is nonnegative for every $X$, and $\delta(X)=0$ iff $X$ is almost surely constant.

Naturally, this also obtains directly through Jensen's inequality applied to $\sqrt{X}$.

And this result is just a special case of the fact that, for every $s$ in $(0,1)$, $$E(\ln X)\leqslant [\ln E(X)]-\delta_s(X)\quad\mbox{where}\ \delta_s(X)=\ln[E(X)/E(X^s)^{1/s}].$$ The quantity $\delta_s(X)$ is a nonincreasing function of $s$ hence the upper bound $[\ln E(X)]-\delta_s(X)$ is better and better when $s$ decreases to $0$. For every $X$, $\delta_1(X)=0$, $\delta_{1/2}(X)=\delta(X)$ and $\delta_0(X)=[\ln E(X)]-E(\ln X)$.

The interesting point in all this, if any, is that one has quantified the discrepancy between $E(\ln X)$ and $\ln E(X)$ and, simultaneously, recovered the fact that $E(\ln X)=\ln E(X)$ iff $X$ is almost surely constant.

• I didn't expect such a complete and neat answer in such a short time... – Shai Covo Feb 13 '11 at 17:05
• @Shai: Well, once you had pointed me in the right direction... Thanks again for the reference to Aldaz's paper. – Did Feb 13 '11 at 20:09
• Can you provide a mathematical definition for "$X$ is almost surely constant"? – NalRa Feb 4 '15 at 11:12
• @NalRa Yes I can. And you? – Did Feb 4 '15 at 11:13
• @NalRa If this is your problem, I suggest to ask it as a separate question instead of obliquely (and rather incomprehensibly) commenting on old unrelated answers. – Did Feb 4 '15 at 11:23

To add on Didier's answer, it is instructive to note that the inequality ${\rm E}(\ln X) \le \ln {\rm E}(X)$ can be seen as a consequence of the AM-GM inequality combined with the strong law of large numbers, upon writing the AM-GM inequality $$\sqrt[n]{{X_1 \cdots X_n }} \le \frac{{X_1 + \cdots + X_n }}{n}$$ as $$\exp \bigg(\frac{{\ln X_1 + \cdots + \ln X_n }}{n}\bigg) \le \frac{{X_1 + \cdots + X_n }}{n},$$ and letting $n \to \infty$.

EDIT: For completeness, let me note that ${\rm E}[\ln X]$ might be equal to $-\infty$. For example, if $X$ has density function $$f(x) = \frac{{\ln a}}{{x\ln ^2 x}},\;\;0 < x < \frac{1}{a},$$ where $a>1$ (note that $\int f = 1$), then $${\rm E}[\ln X] = \int_0^{1/a} {\frac{{\ln a}}{{x\ln x}}} \,{\rm d}x = -\infty.$$

• Nice. I have always wondered whether AM-GM also proved the equality case, maybe you know the answer. – Did Feb 12 '11 at 13:49
• @Didier: Interesting question. It seems that a partial answer can be deduced from Theorem 1 in files.ele-math.com/articles/jmi-03-21.pdf (by letting $\alpha_i = 1/n$). – Shai Covo Feb 13 '11 at 13:02
• Thanks, I added an explanation of this to my answer. – Did Feb 13 '11 at 15:08