25
$\begingroup$

Assuming I have a always positive random variable $X$, $X \in \mathbb{R}$, $X > 0$. Then I am now interested in the difference between the following two expectation values:

  1. $E \left[ \ln X \right]$
  2. $\ln E \left[ X \right]$

Is one maybe always a lower/upper bound of the other?

Many thanks in advance...

$\endgroup$
36
$\begingroup$

Since the function logarithm is concave, Jensen's inequality shows that $E(\ln(X))\leqslant \ln E(X)$ and that equality occurs iff $X$ is almost surely constant.


Edit (This is to expand on a remark made by Shai.)

Shai's answer explains how to prove $E(\ln(X))\leqslant \ln E(X)$ using only AM-GM inequality and the strong law of large numbers. These very tools yield the following refinement (adapted from the paper Self-improvement of the inequality between arithmetic and geometric means by J. M. Aldaz).

Apply AM-GM inequality to the square roots of an i.i.d. sequence of positive random variables $(X_i)$, that is, $$ \sqrt[n]{\sqrt{X_1}\cdots\sqrt{X_n}}\leqslant\frac1n(\sqrt{X_1}+\cdots+\sqrt{X_n}). $$ In the limit $n\to\infty$, the strong law of large numbers yields $$ \exp(E(\ln\sqrt{X}))\leqslant E(\sqrt{X}), $$ that is, $$ E(\ln X)\leqslant 2\ln E(\sqrt{X})=\ln (E(X)-\mbox{var}(\sqrt{X})). $$ Finally:

For every positive integrable $X$, $$ E(\ln X)\leqslant [\ln E(X)]-\delta(X)\quad\mbox{where}\ \delta(X)=\ln[E(X)/E(\sqrt{X})^2]. $$

The correction term $\delta(X)$ is nonnegative for every $X$, and $\delta(X)=0$ iff $X$ is almost surely constant.

Naturally, this also obtains directly through Jensen's inequality applied to $\sqrt{X}$.

And this result is just a special case of the fact that, for every $s$ in $(0,1)$, $$ E(\ln X)\leqslant [\ln E(X)]-\delta_s(X)\quad\mbox{where}\ \delta_s(X)=\ln[E(X)/E(X^s)^{1/s}]. $$ The quantity $\delta_s(X)$ is a nonincreasing function of $s$ hence the upper bound $[\ln E(X)]-\delta_s(X)$ is better and better when $s$ decreases to $0$. For every $X$, $\delta_1(X)=0$, $\delta_{1/2}(X)=\delta(X)$ and $\delta_0(X)=[\ln E(X)]-E(\ln X)$.

The interesting point in all this, if any, is that one has quantified the discrepancy between $E(\ln X)$ and $\ln E(X)$ and, simultaneously, recovered the fact that $E(\ln X)=\ln E(X)$ iff $X$ is almost surely constant.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ I didn't expect such a complete and neat answer in such a short time... $\endgroup$ – Shai Covo Feb 13 '11 at 17:05
  • $\begingroup$ @Shai: Well, once you had pointed me in the right direction... Thanks again for the reference to Aldaz's paper. $\endgroup$ – Did Feb 13 '11 at 20:09
  • $\begingroup$ Can you provide a mathematical definition for "$X$ is almost surely constant"? $\endgroup$ – NalRa Feb 4 '15 at 11:12
  • $\begingroup$ @NalRa Yes I can. And you? $\endgroup$ – Did Feb 4 '15 at 11:13
  • 1
    $\begingroup$ @NalRa If this is your problem, I suggest to ask it as a separate question instead of obliquely (and rather incomprehensibly) commenting on old unrelated answers. $\endgroup$ – Did Feb 4 '15 at 11:23
10
$\begingroup$

To add on Didier's answer, it is instructive to note that the inequality ${\rm E}(\ln X) \le \ln {\rm E}(X)$ can be seen as a consequence of the AM-GM inequality combined with the strong law of large numbers, upon writing the AM-GM inequality $$ \sqrt[n]{{X_1 \cdots X_n }} \le \frac{{X_1 + \cdots + X_n }}{n} $$ as $$ \exp \bigg(\frac{{\ln X_1 + \cdots + \ln X_n }}{n}\bigg) \le \frac{{X_1 + \cdots + X_n }}{n}, $$ and letting $n \to \infty$.

EDIT: For completeness, let me note that ${\rm E}[\ln X]$ might be equal to $-\infty$. For example, if $X$ has density function $$ f(x) = \frac{{\ln a}}{{x\ln ^2 x}},\;\;0 < x < \frac{1}{a}, $$ where $a>1$ (note that $\int f = 1$), then $$ {\rm E}[\ln X] = \int_0^{1/a} {\frac{{\ln a}}{{x\ln x}}} \,{\rm d}x = -\infty. $$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Nice. I have always wondered whether AM-GM also proved the equality case, maybe you know the answer. $\endgroup$ – Did Feb 12 '11 at 13:49
  • $\begingroup$ @Didier: Interesting question. It seems that a partial answer can be deduced from Theorem 1 in files.ele-math.com/articles/jmi-03-21.pdf (by letting $\alpha_i = 1/n$). $\endgroup$ – Shai Covo Feb 13 '11 at 13:02
  • $\begingroup$ Thanks, I added an explanation of this to my answer. $\endgroup$ – Did Feb 13 '11 at 15:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.