1
$\begingroup$

https://www2.warwick.ac.uk/fac/cross_fac/complexity/study/msc_and_phd/co904/co904online/lecture-3-4.pdf (the without expectation setting) \begin{align*} maximize \ &\sum_{i=1}^K -p_i log(p_i) \\ \ subject \ to \ &\sum_{i=1}^K p_i =1 \end{align*} $p_i$ is the probability distribution over a set $\left\{x_i\right\}_{i=1}^K$.

I was just wondering: we have $\sum_{i=1}^N p_i =1$ (for a discrete probability distribution when we are trying to solve a maximum entropy problem) as a constraint. Is the correct reason as to why we don't need $p_i \ge 0 \ \forall i$ as another constraint that this is already captured as an implicit constraint in the $log(.)$ function (with a strict inequality, in fact)?

$\endgroup$
1
$\begingroup$

The assumptions $p_i\geq 0$ must always be present, but depending on your setup it could be buried in some other assumptions. Can't say for sure unless you write out the full setup and optimization problem you are considering.

$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.