Solution to variational problem I am struggling to see how to get to the solution of the equation below. The problem equation and solution come from here (see eqn 3 and eqn4). They state that this is a variational problem,
$$
p^{*}(a | w)=\underset{p(a | w)}{\arg \max } \sum_{a} \left[ p(a | w) U(w, a)-\frac{1}{\beta}  p(a | w) \log \frac{p(a | w)}{p_{0}(a)} \right]
$$
given that $0<p(a | w)<1$ and $\sum_{a} p(a | w) =1$
The answer is:
$$
p^{*}(a | w)=\frac{1}{Z(w)} p_{0}(a) e^{\beta U(w, a)}
$$
where $Z$ is the normalisation constant
$$
Z(w)=\Sigma_{a} p_{0}(a) e^{\beta U(w, a)}
$$
Edit: Added the constraint that p lies between 0 and 1 (as it's a probability)
 A: The problem can restated more clearly as looking for $$\text{argmax}_{x\in \mathbb R^n} \sum_{i=1}^n x_i U_i -\frac 1\beta x_i\log\frac{x_i}{y_i} \quad \text{such that }\sum_{i=1}^n x_i=1 $$
Note that the constraints $x_i\geq 0$ are implicit since $\log x_i$ appears in the objective. The Lagrangian is $$\mathcal L(x,\gamma) = \sum_{i=1}^n x_i U_i -\frac 1\beta x_i\log\frac{x_i}{y_i} + \gamma ( \sum_{i=1}^n x_i-1)$$
and $\displaystyle \frac{\partial \mathcal L}{\partial x_i}(x,\gamma) = U_i-\frac 1\beta \left( \log\frac{x_i}{y_i}+1\right)+\gamma$, which is $0$ if and only if $\displaystyle y_ie^{\beta U_i}e^{\gamma-1}=1$.
The constraint on the $x_i$ implies $e^{\gamma-1}= \sum_{i=1}^n y_ie^{\beta U_i}$, so that the optimal $x$ is given by $$x_i= \frac{y_ie^{\beta U_i}}{\sum_{i=1}^n y_ie^{\beta U_i}}$$
A: There's one extra constraint you've omitted—namely
$$
\sum_a x_a = 1\ .
$$
The Lagrangian function $\ \mathcal{L}\ $for the optimisation problem, then, is given by
\begin{eqnarray}
\mathcal{L}(x,\lambda) &=& \sum_a \left[x_a U(w,a) -\frac{x_a}
{\beta}\log\frac{x_a}{p_0(a)}\right]-\lambda\left(\sum_ax_a-1\right)\\
&=& \lambda + \sum_a x_a\left[ U(w,a) + \frac{\log p_0(a)}{\beta}-\lambda-\frac{\log x_a}{\beta}\right]\ ,
\end{eqnarray}
and the first-order optimality conditions are
\begin{eqnarray}
0&=&\frac{\partial L}{\partial x_a}(x,\lambda)\\
 &=& \left[ U(w,a) + \frac{\log p_0(a)}{\beta}-\lambda-\frac{\log x_a}{\beta}\right]-\frac{1}{\beta}\ \ \ \mbox{for all $\ a\ $.}
\end{eqnarray}
These give
$$
x_a^*=p_0(a)e^{\beta U(w,a)-\beta\lambda-1}\ ,
$$
and plugging these values into the constraint $\ \sum_a x_a = 1\ $ gives
$$
e^{\beta\lambda+1}=\sum_a p_0(a)e^{\beta U(w,a)}=Z(w)\ ,
$$
and so
$$
x_a^*=\frac{p_0(a)e^{\beta U(w,a)}}{Z(w)}\ ,
$$
as given in the paper cited.
The second partial derivatives of the objective, which are just the same as $\ \frac{\partial^2 \mathcal{L}}{\partial x_a\partial x_b}\ $ in this case, are given by
$$
\frac{\partial^2 \mathcal{L}}{\partial x_a\partial x_b}(x,\lambda) = -\frac{\delta_{ab}}{\beta x_a}< 0 ,
$$
 provided $\ \beta>0\ $. In that case  the Hessian is negative definite over the whole domain of the objective,  the objective is strictly concave, and $\ x^*\ $ is the unique global maximiser.   If, instead, $\ \beta < 0\ $, the objective function would be strictly convex, and  $\ x^*\ $ would be the global minimiser.
