Entropy expression optimization with Langrange multipliers I have recently encountered variants of the following expression:
\begin{equation}
S = H(a,b,c,d)-H(a+b,c+d)
\end{equation}
where $H$ is the Shannon entropy function, that is $H(X)=\sum_{x\in X}-x\log x$. And restrictions:
\begin{eqnarray}
a + b &=& t\\
a + c &=& t\\
a + b +c + d &=& 1 
\end{eqnarray}
for some given $t$. It is supposed to be easy to maximize this expression with Lagrange multipliers but I am unfamiliar with them so any hint in how $S$ can be optimized would be welcome.
 A: Presumably $a,b,c,d > 0$ are required.  In order for this to be possible, $0 < t < 1$.
You  should actually check the limits as one or more of $a,b,c,d$ go to $0$, but I will not do so.   Putting in a Lagrange multiplier $\lambda_i$ for each constraint, the Lagrangian is
$$L = -a\ln  \left( a \right) -b\ln  \left( b \right) -c\ln  \left( c
 \right) -d\ln  \left( d \right) + \left( a+b \right) \ln  \left( a+b
 \right) + \left( c+d \right) \ln  \left( c+d \right) +\lambda_{{1}}
 \left( a+b-t \right) +\lambda_{{2}} \left( a+c-t \right) +\lambda_{{3
}} \left( a+b+c+d-1 \right)$$
Now solve the system of equations obtained by setting to $0$ the derivative of $L$ with respect to each of the variables $a,b,c,d,\lambda_1, \lambda_2,\lambda_3$:
$$ \eqalign{ a+b-t &= 0\cr a+c-t &= 0\cr a+b+c+d-1 &= 0\cr -\ln(d)+\ln(c+d)+\lambda_3 &=0\cr -\ln(b)+\ln(a+b)+\lambda_1+\lambda_3 &= 0\cr -\ln(c)+\ln(c+d)+\lambda_2+\lambda_3 &=0\cr  -\ln(a)+\ln(a+b)+\lambda_1+\lambda_2+\lambda_3 &= 0\cr}$$
obtaining
$$ a={t}^{2},b=c=t-{t}^{2}, d=(t-1)^2,\lambda_1 = 0,\lambda_{{2
}}=\ln(t/(1-t)) ,
\lambda_{{3}}=\ln (1-t) $$
A: Another approach is to avoid Lagrange multipliers.
Solve the system of constraints
for $b$, $c$, and $d$ to find
$$\begin{eqnarray*}
b &=& t-a \\
c &=& t-a \\
d &=& a-2t+1.
\end{eqnarray*}$$
We have the restrictions $0\le a,b,c,d \le 1$, so
$\textrm{max}(0,2t-1)\le a\le t$, where $0\le t\le 1$. 
Thus,
$$\begin{equation*}
S(a) = H(a,t-a,t-a,a-2t+1)-H(t,1-t),
\end{equation*}$$
so the extremum is the solution to
$$\frac{\partial}{\partial a} H(a,t-a,t-a,a-2t+1) = 0,$$
or $a = t^2$, reproducing the solution found by @RobertIsrael.
The maximum is $S_{\mathrm{max}} = S(t^2) = (t-1)\log(1-t) - t\log t$.
Note that 
$b=0 \iff c=0 \iff a=t$, but $S(t) = H(t,0,0,1-t)-H(t,1-t) = 0$. 
Also $d=0 \iff a=2t-1.$

Figure 1. Plot of $S_{\mathrm{max}}$ (black), and $S$ for $a=0$ (red) and $d=0$ (blue).
