3
$\begingroup$

Suppose that the Probability Mass Function of a random variable $X$ with values in $A = \{1, 2, \dots\}$ has nonincreasing probabilities, $P(k + 1) \leq P(k)$, for all $k \geq 1$. Show that, if $H(X) < \infty$, then $\mathbb{E}[\log X] < \infty$.

I can intuitively see that this is true since $-\log$ grows very quickly near the origin, much faster than linear. So, if $H(X) = -\sum_{x=1}^\infty p(x)\log p(x)$ is finite, then $p(x)$ must be shrinking quickly enough to modulate this and keep the sum finite. Hence, it decreases quickly enough to keep $\mathbb{E}[\log X] = \sum_{x=1}^\infty p(x)\log x$ finite as well. I can't seem to figure how to put this concept into rigorous math.

$\endgroup$
3
  • 1
    $\begingroup$ so if $\mathbb E (\log X) = \sum_{x=1}^\infty p(x) \log p(x) = - H(X)$, isn't that trivial? $\endgroup$
    – Targon
    Jan 27 at 16:20
  • $\begingroup$ there is a difference between $\log(x)$ and $\log p(x)$... $\endgroup$
    – Eulerian
    Jan 27 at 16:36
  • $\begingroup$ okay, obviously. my bad. $\endgroup$
    – Targon
    Jan 27 at 16:41

2 Answers 2

4
$\begingroup$

This solution is based on the ideas of @Keen-ameteur.

Since the sequence $(p_k)$ is decreasing, we have $p_1 \geq \ldots \geq p_k$ for all $k.$ If, for some $k$ we had that $p_k > \dfrac{1}{k},$ then we would reach that $p_1 + \ldots + p_k > k \dfrac{1}{k} = 1,$ which is impossible. Therefore, for all $k$ we have $p_k \leq \dfrac{1}{k}.$

The result now follows easily since then $-p_k \log p_k \geq p_k \log k.$ Q.E.D.

$\endgroup$
2
$\begingroup$

I think you can apply a comparison test between two series.

Since $p(n)$ is summable and montonically decreasing, we know that $p(n)<\frac{1}{n}$ for $n$ large, and this implies that $-\log(p(n))>\log(n)$. Hence, $-log(p(n))p(n)\geq \log(n)p(n)$ for $n$ large enough.

$\endgroup$
2
  • $\begingroup$ You could have $p(n) > 1/n$ for infinitely many $n$ and the series $p(n)$ be summable. Also notice that you never used that $p(n)$ is decreasing... $\endgroup$
    – William M.
    Jan 27 at 17:39
  • $\begingroup$ You are right, I'll edit that. @WilliamM. $\endgroup$ Jan 27 at 17:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.