Things I understand:
- is the expected amount of information in an event from that distribution.
- In game of 20 questions to guess an item, it is the lower bound on the number questions one could ask.
It gives the lower bound on the number of bits needed on average to encode symbols drawn from a distribution.
I don't understand the why it mentions on average. Isn't it just a lower bound. Also, please elaborate on the lower bound if possible.