I have a basic question about the Shannon Entropy formula. In fact it's so dumb that I didn't dare ask it in the class because I don't understand the text books.

Here's the formula:

$$H(X) = -\displaystyle\sum_{x} p(x)\log p(x).$$

And here's my question: what function is p(x)? Is it frequency, and if so, frequency of what?

  • $\begingroup$ As the answer says, $p$ is probability. A reasonable explanation is found in this brochure (pages 2-5): books.google.com/… $\endgroup$ – avs Jul 22 '16 at 5:30

It is a probability mass function. In particular, $p(x_0)$ is the probability of event $x_0$ occurring.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.