How can be proven that the entropy of a dice roll is maximized when the probability of each of its $6$ faces is equal, $1/6$?
Tell me more
×
Mathematics Stack Exchange is a question and answer site for
people studying math at any level and professionals in related fields. It's 100% free, no registration required.
|
Suprise is defined as -log(p{X=x}). A good way to think of entropy is the "expected surprise". In this sense, its easy to see that the uniform distribution maximizes the expected surprise. |
|||
|
|
|
The entropy is given by $-\sum p_i\ln p_i$. Use Jensen's inequality with the logarithm function. |
|||
|
|
