# Shannon entropy understanding

Things I understand:

Shannon entropy-

• is the expected amount of information in an event from that distribution.
• In game of 20 questions to guess an item, it is the lower bound on the number questions one could ask.

Doubt:

It gives the lower bound on the number of bits needed on average to encode symbols drawn from a distribution.

I don't understand the why it mentions on average. Isn't it just a lower bound. Also, please elaborate on the lower bound if possible.

For example, consider a distribution with three values $(A,B,C)$ with probabilities $(1/2, 1/4, 1/4)$. In this case, the optimum sequence of questions is first ask "Is it A"? If no, ask "Is it B"? To discover the element we need sometimes 1 question, sometimes 2 questions. In average, we need $3/2$ questions. Which (you can check) coincides with the entropy. In this case, the bound is achieved.