# Average Shannon Entropy

Suppose we are given a finite discrete random variable $$Y$$ which picks a random variable from the set $$\mathcal{Y} = \{X_1, \dots, X_n \}$$, the $$X_i$$ are finite discrete random variables over the same set $$\mathcal{X}$$, according to some probability distribution $$\mathbb{P}$$. Let $$H(X)$$ denote the usual Shannon entropy.

I'm interested in calculating the average entropy of $$Y$$, that is

$$\mathbb{E}[H(Y)] = \sum_{X \in \mathcal{Y}} \mathbb{P} [X] H(X).$$

Is this quantity known or has it been considered before?

Thank you very much in advance!

To start with, I would define your average entropy as a function of $$X_1,\dots,X_n$$, since $$Y$$ is already the mixture of the $$X_i$$'s. So the average entropy, as you define it, cannot be expressed as a function of the density of $$Y$$.
Regarding your question, I believe the most meaningful interpretation of your quantity is the following. Denote by $$I$$ a random variable that takes values in $$\{1,2,\dots,n\}$$ and is distributed according to $$\mathbb{P}$$ and independent from $$X_1,\dots,X_n$$. Then, one can express $$Y = X_I$$, and the conditional entropy of $$Y$$ given $$I$$ is
$$H(Y|I) = H(X_I|I) = \sum_{i=1}^n P(I=i) H(X_I|I=i) = \sum_{i=1}^n P(I=i) H(X_i).$$ This is precisely your average entropy term. This means, the average entropy is simply the entropy of $$Y$$ conditioned on the pick $$I$$.