I am reading the proof of Shannons Entropy in his paper of 1948. (http://math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf)

In the proof (Aprendix 2, page 29), I do not understand why

$|\frac{A(t)}{A(s)}-\frac{\log t}{\log s}|<2\epsilon$

implies that

$ A(t)=K \log t$

Where does the $K$ came from?


Since $\epsilon$ is arbitrarily small, Shannon has proved that

$\frac{A(t)}{A(s)} = \frac{\log(t)}{\log(s)} \quad \forall s,t \in \mathbb{Z}^+$

Re-arranging this identity gives

$\frac{A(t)}{\log(t)} = \frac{A(s)}{\log(s)} \quad \forall s,t \in \mathbb{Z}^+$

If we call this ratio $K$ then we have

$\frac{A(t)}{\log(t)} = K \quad \forall t \in \mathbb{Z}^+$

$\Rightarrow A(t) = K \log(t)\quad \forall t \in \mathbb{Z}^+$

  • 1
    $\begingroup$ $s$ and $t$ are arbitrary positive integers so it doesn't matter whether we use $s$ or $t$ - it is a bound variable. I have made an edit to clarify this. $\endgroup$ – gandalf61 Jun 4 '18 at 14:25
  • $\begingroup$ Thank you very much! I got it! :) $\endgroup$ – Patricio Jun 7 '18 at 14:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.