2
$\begingroup$

Given a multinomial distribution:

$Mul(\overline{x}|n, \overline{p}) = {n \choose x_{1...k}!}\prod_{i}^{k}p_i^{x_i} $

And the entropy of a probability distribution:

$Entropy(x) = -\sum_i^n p_x(x)log(p_x(x))$

How do we maximize entropy when it comes to multinomial w.r.t $\overline{p}$? It's simple if we have a binomial example, which is p and 1-p, and just solving for p when the derivative of the entropy with respect to p is equal to 0(or if we can't find a maxima, then use lagrange multipliers with the bound shown next. However, in this case we have to satisfy a bound on maximizing entropy:

$\sum_i^k p_i = 1$

Any hints would be appreciated!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.