Finding the maximum of a function using Lagrange multipliers Consider the function $x^a e^{-x} y^b e^{-y}$, over the triangle defined by $x \ge 0, y \ge 0, x + y = 1$. 
How can one find the maximum of such a function in terms of $a,b \ge 0$? This deeply confuses me, since the triangle is not smooth...
 A: Define
$$H(x,y,\lambda):=x^ae^{-x}y^be^{-y}+\lambda(x+y-1)\Longrightarrow$$
$$\begin{align*}H'_x&=y^be^{-y}x^{a-1}e^{-x}\left(a-x\right)+\lambda=0\Longrightarrow \lambda=-y^be^{-y}x^{a-1}e^{-x}\left(a-x\right)\\H'_y&=x^ae^{-x}y^{b-1}e^{-y}\left(b-y\right)+\lambda=0\Longrightarrow \lambda=-x^ae^{-x}y^{b-1}e^{-y}\left(b-y\right)\\H'_\lambda&=x+y-1=0\end{align*}$$
Compairng $\,H'_x\,,\,H'_y\,$ we get
$$x^ae^{-x}y^{b-1}e^{-y}\left(b-y\right)=y^be^{-y}x^{a-1}e^{-x}\left(a-x\right)\Longleftrightarrow x(b-y)=y(a-x)$$
but $\,H'_\lambda\,$ tells us that $\,y=1-x\,$ , so
$$x(b+x-1)=(1-x)(a-x)\Longleftrightarrow x(b-1)=a-(a+1)x\ldots$$
Try to take it from here, and remember: $\,x,y\ge 0\,$
A: Why do you bother about smoothness of domain set limits? The main focus is the function itself.
If you don't want to use Lagrange multiplier method you can use variational calculus directly:
$$max_{x,y}F(x,y)\quad s.t.\ G(x,y)=c$$
Starting with constraint $G(x,y)=c$ must hold all the time. This will be the first equation.
Using implicit function theorem we can derive that $y=y(x)$. By strenghening the constraint and using $y=y(x)$
$$dG(x,y)=\frac{\partial G}{\partial x}dx+\frac{\partial G}{\partial y}\frac{dy}{dx}dx=0$$
$$\Rightarrow\ \frac{dy}{dx}=-\frac{\frac{\partial G}{\partial x}}{\frac{\partial G}{\partial y}} $$
Using implicit function theorem again and equating total derivative of $F(x,y)$ to zero
$$dF(x,y)=\frac{\partial F}{\partial x}dx+\frac{\partial F}{\partial y}*\frac{dy}{dx}dx=0$$
and substitiuting $\frac{dy}{dx}$
$$dF(x,y)=\bigg(\frac{\partial F}{\partial x}-\frac{\partial F}{\partial y}*\frac{\frac{\partial G}{\partial x}}{\frac{\partial G}{\partial y}}\bigg)dx=0$$
The system of equations to be solved are
$$[1]\qquad G(x,y)=c$$
$$[2]\qquad \frac{\partial F}{\partial x}-\frac{\partial F}{\partial y}*\frac{\frac{\partial G}{\partial x}}{\frac{\partial G}{\partial y}}=0$$
$\mathbb PS1:$ You would come up with the same system if you would have used LM method and eliminated $\lambda$
$\mathbb PS2:$ For your specific question $x=\frac{a}{a+b}\ y=\frac{b}{a+b}$ which are valid since $a,b>0\Rightarrow\ x,y>0$
