I have set of linear constraints and would like to find an answer to its unknown variables, $p_i$'s. One of my options to find a solution for $p_i$'s using maximum entropy problem, $\max(\sum - p_i \log p_i)$. However the condition $\sum p_i=1$ does not hold. I would like to know how necessary this condition is?

I have given this problem to a solver and it converges after a few iteration, however if I divide $p_i$ by a constant to satisfy the $\sum p_i=1$ the solver fails.

  • 2
    $\begingroup$ How can $p_i$ represent a probability distribution if they do not sum to $1$? Can you say a little more about your model? $\endgroup$
    – user856
    Commented May 30, 2014 at 22:57
  • $\begingroup$ Actually $p_i$'s are variables, so let's not call a probability distribution. $\endgroup$
    – HHH
    Commented May 30, 2014 at 23:05
  • $\begingroup$ So why do you want $\sum p_i =1$ if the $p_i$'s are variables? $\endgroup$ Commented May 31, 2014 at 19:58
  • $\begingroup$ You should mention that this is a continuation of your efforts in this question. It's not a duplicate, but the relationship should still be noted. $\endgroup$ Commented Jun 1, 2014 at 19:23
  • $\begingroup$ The fact is that $\sum_i p_i = 1$ is just another linear equality constraint on the variables, like all of the others. As with any set of constraints, they are either feasible or infeasible. It sounds like your original model is feasible, but adding $\sum_i p_i=1$ makes it infeasible. It's as simple as that. Whether it is critical that $\sum_i p_i=1$ hold is up to you to decide. We can't help you there. $\endgroup$ Commented Jun 1, 2014 at 19:24


You must log in to answer this question.