0
$\begingroup$

I'm studying information theory, and working through this document. On page 17, it shows that, with the function that gets the entropy of a probability $I$ and a probability $p$, that $I(p^a) = a * I(p)$. I can follow how this was derived from the axioms given, however I can't understand why $I(p^a) = a * I(p)$ means $I(p) = -log_b(p) = log_b(1/p)$. Could someone explain this?

|cite|improve this question
$\endgroup$
2
$\begingroup$

$\log_{b}(p^a)=a\log_{b}(p) $.
There are no other solutions over the positive reals because given any positive $x,c$ with $I(x)=c$ and $x\not=1$ all positive real numbers $r$ can be expressed uniquely as a power of $x$ and $I(r)$ is then uniquely determined.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Sorry, I still don't quite understand. $log_b(p^a) = a*log_b(p)$ is the same as the previously deduced $I(p^a) = a*I(p)$, so I could see why $I(p)$ could equal $log_b(p)$, but why does it equal $log_b(1/p)$? $\endgroup$ – rlms Aug 4 '13 at 18:06
  • $\begingroup$ $log_b(\frac{1}p)=log_{\frac{1}b}(p)$ $\endgroup$ – Angela Richardson Aug 5 '13 at 5:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.