I'm studying information theory, and working through this document. On page 17, it shows that, with the function that gets the entropy of a probability $I$ and a probability $p$, that $I(p^a) = a * I(p)$. I can follow how this was derived from the axioms given, however I can't understand why $I(p^a) = a * I(p)$ means $I(p) = -log_b(p) = log_b(1/p)$. Could someone explain this?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
$\log_{b}(p^a)=a\log_{b}(p) $.
There are no other solutions over the positive reals because given any positive $x,c$ with $I(x)=c$ and $x\not=1$ all positive real numbers $r$ can be expressed uniquely as a power of $x$ and $I(r)$ is then uniquely determined.
answered Aug 4, 2013 at 16:12
-
$\begingroup$ Sorry, I still don't quite understand. $log_b(p^a) = a*log_b(p)$ is the same as the previously deduced $I(p^a) = a*I(p)$, so I could see why $I(p)$ could equal $log_b(p)$, but why does it equal $log_b(1/p)$? $\endgroup$– rlmsAug 4, 2013 at 18:06
-
$\begingroup$ $log_b(\frac{1}p)=log_{\frac{1}b}(p)$ $\endgroup$ Aug 5, 2013 at 5:48