1
$\begingroup$

Given the joint probability distribution of a set of non-negative random variables $y_1,\dots y_N$, is there an analytic way to calculate the probability distribution of the mixture entropy $S$, defined as $$ S=-\sum_{i=1}^N \frac{y_i}{Y}\,\log \frac{y_i}{Y} $$ where $Y=\sum_{i=1}^N y_i$? If this is not possible in general, is it possible when $y_1,\dots y_N$ are i.i.d.? Or perhaps when they are negative binomial in distribution, $y_i\sim\text{NBin}(r_i,p)$ with equal $p$?

$\endgroup$
  • $\begingroup$ That looks rather hopeless to me.. even if the $y_i$ are iid, $Y$ is still a random variable, and $y_i/Y$ are not iid... At most one can perhaps find some approximation for large $N$... $\endgroup$ – leonbloy May 16 '17 at 22:35
  • $\begingroup$ Thanks for your comment! Perhaps a more productive question would be this: Given the joint probability distribution of a set of non-negative random variables $x_1,\dots x_N$ that are known to sum to 1, is there an analytic way to calculate the probability distribution of the mixture entropy $S$, defined as $$ S=-\sum_{i=1}^N x_i\log x_i $$ $\endgroup$ – Alex May 16 '17 at 23:58
  • $\begingroup$ Yes, but that's about the same. (and the $x_i$ cannot be iid). $\endgroup$ – leonbloy May 17 '17 at 0:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.