# Maximum entropy problem constraints

https://www2.warwick.ac.uk/fac/cross_fac/complexity/study/msc_and_phd/co904/co904online/lecture-3-4.pdf (the without expectation setting) \begin{align*} maximize \ &\sum_{i=1}^K -p_i log(p_i) \\ \ subject \ to \ &\sum_{i=1}^K p_i =1 \end{align*} $p_i$ is the probability distribution over a set $\left\{x_i\right\}_{i=1}^K$.

I was just wondering: we have $\sum_{i=1}^N p_i =1$ (for a discrete probability distribution when we are trying to solve a maximum entropy problem) as a constraint. Is the correct reason as to why we don't need $p_i \ge 0 \ \forall i$ as another constraint that this is already captured as an implicit constraint in the $log(.)$ function (with a strict inequality, in fact)?

The assumptions $p_i\geq 0$ must always be present, but depending on your setup it could be buried in some other assumptions. Can't say for sure unless you write out the full setup and optimization problem you are considering.

• The problem would be maximize $\sum_{i=1}^K -p_i log(p_i)$ subject to $\sum_{i=1}^K p_i =1$. $p_i$ is the probability distribution over a set $\left\{x_i\right\}_{i=1}^K$. – learning Mar 21 '17 at 5:59
• – learning Mar 21 '17 at 6:38