0
$\begingroup$

In the source article at the bottom, the antilog of entropy is introduced as the exponential of Shannon entropy:

$$ H(N) = -\sum_N w_i \ln w_i $$

$$ e^{H(N)} = \prod_N w_i ^{w_i} $$

  1. Could someone show the steps in between $e^{H(N)}$ and how to get to $\prod_N w_i ^{w_i} $?
  2. and can the last measure also be applied to point-wise probabilities of a random variable like how Shannon entropy normally is used for, i.e. $p(x)$ instead of $w_i$ so that, for random variable $X$,

$$e^{H(X)} = \prod_N p(x) ^{p(x)}?$$

Straathof, S. M. (2007). Shannon's entropy as an index of product variety. Economics Letters, 94(2), 297-303.

$\endgroup$
3
  • 1
    $\begingroup$ By the laws of indices/logarithms, $$\exp H(N)= \exp\Big(-\sum_{N} w_i\log w_i\Big) = \prod_{N} \exp(-w_i\log w_i) = \prod_{N} \exp(\log(w_i^{-w_i})) = \prod_N {w_i}^{-w_i},$$ since $\exp$ and $\log$ are inverses of each other. $\endgroup$ Commented Nov 10, 2020 at 22:36
  • $\begingroup$ thanks, could you change the comment to an answer $\endgroup$
    – develarist
    Commented Nov 10, 2020 at 22:37
  • 1
    $\begingroup$ @deverlarist I did it as a comment cause I didn't really answer your second point. $\endgroup$ Commented Nov 10, 2020 at 22:40

1 Answer 1

2
$\begingroup$

By the laws of indices/logarithms, $$\exp H(N)= \exp\Big(-\sum_{N} w_i\log w_i\Big) = \prod_{N} \exp(-w_i\log w_i) = \prod_{N} \exp(\log(w_i^{-w_i})) = \prod_N {w_i}^{-w_i},$$ since $\exp$ and $\log$ are inverses of each other.

$\endgroup$
3
  • $\begingroup$ the question was edited with a second question if you're up for it $\endgroup$
    – develarist
    Commented Nov 10, 2020 at 22:40
  • $\begingroup$ can the laws of indices/logarithms also be used to produce the same result if Shannon entropy were replaced with differential entropy: $$h(X) = \int f(x) \log f(x) dx ?$$ $\endgroup$
    – develarist
    Commented Nov 10, 2020 at 23:15
  • 1
    $\begingroup$ @develarist Not easily, no. There is no equivalent to $\exp\big(\sum{}\cdot{}\big) = \prod\exp({}\cdot{})$ for integrals. $\endgroup$ Commented Nov 10, 2020 at 23:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .