# How to perform a constrained optimisation of a log likelihood function

I have 3 binomial random variables $$x_1, x_2, x_3$$ with parameters $$\theta_1, \theta_2, \theta_3$$

If I have a log likelihood function $$L = \sum_{i=1}^{3} (\log(^{N_i} C_{x_i} \times \theta_i^{x_i} \times(1-\theta_i)^{N_i-x_i} )$$

I need to maximize this function subject to constrain that $$\theta_1 < \theta_2 < \theta_3$$

Is this something which can be done using some known method like simulations or Bayesian approaches? I know about simulated annealing method but how do I make constraints in that algorithm?

• what about solving it without constraint and depending on the result, relaunching the optimization after having merged the $\theta_i$ that are not in the correct order relatively to each other ? Commented Sep 22, 2021 at 10:36
• @ArnaudMégret Obviously the MLE of unconstrained log likelihood is just x/N . By merging you mean merging groups 1 and 2 or 2 and 3 into a single group? Commented Sep 22, 2021 at 10:59
• Yes, I consider seeing that the data of, for instance, $x_1$ and $x_2$ come from the same variable. My intuition is that if MLE withoust constraint gives the estimations $\theta_1' \gt \theta_2'$ then enforcing the constraint $\theta_1' \le \theta_2'$ should lead to new estimations where $\theta_1' = \theta_2'$ Commented Sep 22, 2021 at 11:34
• @ArnaudMégret Anyway to not merge this and achieve the result? So my logic is that the observed θs (which are not monotonous) are not far away from the actual θs (unobserved) which follow the monotonic trend. Commented Sep 22, 2021 at 14:49
• I am not sure to understand your point. I think that by merging, you will achieve your MLE optimization. If you find $\theta_1' \gt \theta_2'$, by merging you will find a estimation in the between for both $\theta_1$ and $\theta_2$. That sounds natural. Commented Sep 22, 2021 at 15:25

You are effectively maximizing $$\sum_{i=1}^3 x_i log(\theta_i) + (N_i-x_i)\log(1-\theta_i)$$ which is convex in $$\theta$$. Hence any standard nonlinear solver should work fine, and if you want to be extra fancy you can use a dedicated exponential-cone solver.