# Max the entropy

I have a simple optimization problem, but somewhere I am making a mistake. I want to maximize the entropy for the four $p_1, p_2, p_3, p_4$. I want to use Lagrange multiplier with constraints. Thus:

$\sum_i p_i \log_2(p_i)$ for $i$ of the interval $[1,4]$
s.t.
$g(p_1,p_2,p_3,p_4) = p_1+p_2+p_3+p_4 = 1$ .

Now I want to compute it with Lagrange multiplier: $$L(p_1,p_2,p_3,p_4,\lambda)= -p_1 \log p_1 - p_2 \log p_2 - p_3 \log p_3 - p_4 \log p_4- \lambda(p_1 + p_2 + p_3 + p_4 - 1)$$ If I compute the partial derivatives:
$$\frac{\partial L}{\partial p_i} = -\log p_i - 1 - \lambda= 0$$ Thus:
$$p_i = - e - e^{\lambda}$$ If I now insert it in the Lagrange function $L$:
$$L(p_1,p_2,p_3,p_4,\lambda)= -p_1 \log p_1 - p_2 \log p_2 - p_3 \log p_3 - p_4 \log p_4- \lambda(p_1 + p_2 + p_3 + p_4 - 1)$$ I will have negative values in the log:
$$-(- e - e^{\lambda} \log(-e-e^{\lambda}))$$ which is not solvable.
I don't know where my mistake is.

• Note that it makes no sense to use Lagrange multipliers for this, because $p$ isn't differentiable (it's non-zero only at four points!). I've asked this question in the past and will link to it. – Clarinetist May 1 '18 at 11:45
• @Clarinetist This problem is defined on a finite set with cardinality 4, not the same as your problem :) – Xiangxiang Xu May 1 '18 at 12:07

Since $\frac{\partial L}{\partial p_i} = -\log p_i - 1 - \lambda = 0$, we have $$p_i = \mathrm{e}^{-\lambda - 1}.$$ As a result, $$p_1 = p_2 = p_3 = p_4 = \frac{1}{4},$$