# Entropy of a mixture given its joint distribution

Given the joint probability distribution of a set of non-negative random variables $y_1,\dots y_N$, is there an analytic way to calculate the probability distribution of the mixture entropy $S$, defined as $$S=-\sum_{i=1}^N \frac{y_i}{Y}\,\log \frac{y_i}{Y}$$ where $Y=\sum_{i=1}^N y_i$? If this is not possible in general, is it possible when $y_1,\dots y_N$ are i.i.d.? Or perhaps when they are negative binomial in distribution, $y_i\sim\text{NBin}(r_i,p)$ with equal $p$?

• That looks rather hopeless to me.. even if the $y_i$ are iid, $Y$ is still a random variable, and $y_i/Y$ are not iid... At most one can perhaps find some approximation for large $N$... – leonbloy May 16 '17 at 22:35
• Thanks for your comment! Perhaps a more productive question would be this: Given the joint probability distribution of a set of non-negative random variables $x_1,\dots x_N$ that are known to sum to 1, is there an analytic way to calculate the probability distribution of the mixture entropy $S$, defined as $$S=-\sum_{i=1}^N x_i\log x_i$$ – Alex May 16 '17 at 23:58
• Yes, but that's about the same. (and the $x_i$ cannot be iid). – leonbloy May 17 '17 at 0:16