# Apply the Lagrange multiplier rule to find the minimizer of an integral functional over a convex set

I want to minimize $$F(w):=\sum_{i\in I}\sum_{j\in I}\int\lambda({\rm d}x)w_i(x)p(x)\int_{\left\{\:pq_j\:>\:0\:\right\}}\lambda({\rm d}y)\frac{\left|w_j(y)p(y)\right|^2}{q_j(y)\sigma_{ij}(x,y)}\left|\frac{f(y)}{p(y)}-\lambda f\right|^2\;\;\;\text{for }w\in L^2(\mu)^I$$ over the set $$C:=\left\{w\in L^2(\mu)^I:\sum_{i\in I}w_i=1\right\}.$$ We easily see that the Fréchet derivative of $$F$$ at $$w$$ is given by $${\rm D}F(w)h=\sum_{i\in I}\sum_{j\in I}\int\lambda({\rm d}x)p(x)\int_{\left\{\:pq_j\:>\:0\:\right\}}\lambda({\rm d}y)\frac{\left|p(y)\right|^2}{q_j(y)\sigma_{ij}(x,y)}\left|\frac{f(y)}{p(y)}-\lambda f\right|^2\left(\left|w_j(y)\right|^2h_i(x)+2w_i(x)w_j(y)h_j(y)\right)\tag1$$ for all $$w,h\in L^2(\mu)^I$$. Using the identificaion $$\mathfrak L(L^2(\mu)^I,\mathbb R)\cong L^2(\mu)^I$$ we may write $$$$\begin{split}&\left({\rm D}F(w)\right)_i(x)\\&\;\;\;\;=\sum_{j\in I}\int_{\left\{\:pq_j\:>\:0\:\right\}}\lambda({\rm d}y)\frac{\left|w_j(y)p(y)\right|^2}{q_j(y)\sigma_{ij}(x,y)}\left|\frac{f(y)}{p(y)}-\lambda f\right|^2\\&\;\;\;\;\;\;\;\;+2\cdot 1_{\left\{\:pq_i\:>\:0\:\right\}}(x)\frac{w_i(x)p(x)}{q_ix)}\left|\frac{f(x)}{p(x)}-\lambda f\right|^2\sum_{j\in I}\int\lambda({\rm d}y)\frac{w_j(y)p(y)}{\sigma_{ij}(x,y)}\end{split}\tag2$$$$ for all $$i\in I$$, $$x\in E$$ and $$w\in L^2(\mu)^I$$.

How can we apply the Lagrange multiplier rule and determine the minimizer $$w$$?

My first problem is that I don't know how I need to incorporate the equality constraint given by the definition of $$C$$.

Definitions:

• $$I$$ is a finite set;
• $$p,q_i$$ are probability densities on a measure space $$(E,\mathcal E,\lambda)$$;
• $$\mu:=p\lambda$$;
• $$f\in\mathcal L^2(\lambda)$$ with $$\{p=0\}\subseteq\{f=0\}$$;
• $$\lambda f:=\int f\:{\rm d}\lambda$$;
• $$w_i:E\to\mathbb R$$ is $$\mathcal E$$-measurable with $$\{q_i=0\}\subseteq\{w_ip=0\}$$ for $$i\in I$$ with $$\{pf\ne0\}\subseteq\left\{\sum_{i\in I}w_i=1\right\}$$;
• $$\sigma_{ij}:E^2\to\mathbb R$$ is $$\mathcal E^{\otimes2}$$-measurable with $$\sigma_{ij}(x,y)=\sigma_{ji}(y,x)$$ for all $$(i,x),(j,y)\in I\times E$$ and $$\sum_{j\in I}\int\lambda({\rm d}y)w_i(x)q_j(y)\sigma_{ij}(x,y)=1$$ for all $$(i,x)\in I\times E$$.

Remark: I'm actually searching for $$\mathcal E$$-measurable $$w_i:E\to\mathbb R$$ with $$\{q_i=0\}\subseteq\{w_ip=0\}$$ for all $$i\in I$$, $$\{pf\ne0\}\subseteq\left\{\sum_{i\in I}w_i=1\right\}$$ and minimizing $$F(w)$$. I guess the description above is the best way to formulate this as an optimization problem, but please let me know if you think that I should search for $$w$$ in a different Banach space or use a different set $$C$$ (maybe incorporating some of the other requirements mentioned before).

If $$w$$ is a minimizer then it holds $$F'(w)h = 0$$ for all $$h$$ such that $$\sum_{i\in I}h_i=0$$ (i.e. for all $$h=u-w$$ with $$u\in C$$). To see this, fix $$h$$ with $$\sum_{i\in I}h_i=0$$. Then $$w+th$$ is in $$C$$ for all $$t$$, and $$F(w+th)-F(w)\ge0$$. Dividing by $$t$$ and passing to the limits $$\searrow0$$ and $$t\nearrow0$$ gives the claim.
Define the mapping $$T: (L^2)^I \to L^2$$ by $$Th = \sum_{i\in I}h_i.$$ Now the optimality condition above implies $$F'(w) \in N(T)^\perp$$ (where $$N$$ is null-space). By the closed range theorem, $$F'(w) \in R(T^*)$$, i.e., there is $$z\in L^2$$ such that $$F'(w) = T^*z$$ or equivalently $$F'(w) h = \sum_i (h_i ,z),$$ or $$F'(w)_i = z,$$ where $$F'(w)_i$$ is such that $$F'(w)h = \sum_{i\in I} F'(w)_ih_i$$.
• (a) Am I missing something or doesn't ${\rm D}F(w)h=0$ hold for all $h\in L^2(\mu)^I$ (not only those satisfying $\sum_{i\in I}h_i=0$)? (b) The rest is clear to me except what exactly "$F'(w)_i=z$" means. I guess you're identifying $\mathfrak L(L^2(\mu)^I,\mathbb R)\cong L^2(\mu)^I$, but what is ${\rm D}F(w)$ under this identification precisely? (c) How can we proceed to find a closed form expression of a minimizer? – 0xbadf00d Oct 25 at 17:36
• (a) no, see edit, (b) I use this identification, also see edit, (c) no idea. Is the functional convex? It seems to be cubic in $w$. – daw Oct 25 at 20:12
• (b) So, you use the same identification as I am, right? (c) You mean the functional $F$? Not sure. What can we infer from $({\rm D}F(w))_i=z$ for all $i\in I$? For example, what could we do in the case $I=\left\{1\right\}$ where we trivially know that the solution is $w_1=1$. – 0xbadf00d Oct 26 at 8:14
• (d) Couldn't we minimize the integrand in the definition of $F(w)$ pointwise? This would yield this problem: math.stackexchange.com/q/3409501/47771. – 0xbadf00d Oct 27 at 7:13