Let $P$ be an infinite partition of the interval $[0,1]$. Let $P$ have elements $I_i$ which has Lebesgue measure $m(I_i)$.

Then the entropy of $P$ is defined by $\sum_i -m(I_i)\log m(I_i)$. Can this be infinite?

  • $\begingroup$ Is $m(I_i)$ the measure of the $i$th interval? $\endgroup$ – Semiclassical Jul 25 '14 at 16:59
  • $\begingroup$ yes, thanks for your comment $\endgroup$ – a12345 Jul 25 '14 at 17:03
  • $\begingroup$ Since $\sum m(I_i) = 1$, the question can be equivalently restated as follows: is there a probability distribution on $\Bbb N$ whose entropy is infinite. As far as I understand, the answer is no as follows from here. $\endgroup$ – Ilya Jul 25 '14 at 17:20
  • $\begingroup$ Thanks for the link. What is the "mean" that they are talking about? $\endgroup$ – a12345 Jul 25 '14 at 17:37
  • $\begingroup$ That I'm not sure I understand: the entropy of a discrete distribution does not depend on its underlying space, so you shall be able to assume the mean to be anything. $\endgroup$ – Ilya Jul 25 '14 at 18:22

This paper provides an example of a distribution on $\Bbb N$ which has infinite entropy. Thus, in your case $P$ is any partition satisfying $$ m(I_i) = \frac{1}{\lg(i+1)} - \frac{1}{\lg(i+2)}, \qquad i\in \Bbb N, $$ where $\lg i$ is a logarithm base $2$. For example, $I_1 = [1,\frac1{\lg3})$, $I_2 = [\frac1{\lg 3},\frac12)$, $I_3 = [\frac12,\frac1{\lg5})$ etc.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.