I want to minimize $$F(w):=\sum_{i\in I}\sum_{j\in I}\int\lambda({\rm d}x)w_i(x)p(x)\int_{\left\{\:pq_j\:>\:0\:\right\}}\lambda({\rm d}y)\frac{\left|w_j(y)p(y)\right|^2}{q_j(y)\sigma_{ij}(x,y)}\left|\frac{f(y)}{p(y)}-\lambda f\right|^2\;\;\;\text{for }w\in L^2(\mu)^I$$ over the set $$C:=\left\{w\in L^2(\mu)^I:\sum_{i\in I}w_i=1\right\}.$$ We easily see that the Fréchet derivative of $F$ at $w$ is given by $${\rm D}F(w)h=\sum_{i\in I}\sum_{j\in I}\int\lambda({\rm d}x)p(x)\int_{\left\{\:pq_j\:>\:0\:\right\}}\lambda({\rm d}y)\frac{\left|p(y)\right|^2}{q_j(y)\sigma_{ij}(x,y)}\left|\frac{f(y)}{p(y)}-\lambda f\right|^2\left(\left|w_j(y)\right|^2h_i(x)+2w_i(x)w_j(y)h_j(y)\right)\tag1$$ for all $w,h\in L^2(\mu)^I$. Using the identificaion $\mathfrak L(L^2(\mu)^I,\mathbb R)\cong L^2(\mu)^I$ we may write \begin{equation}\begin{split}&\left({\rm D}F(w)\right)_i(x)\\&\;\;\;\;=\sum_{j\in I}\int_{\left\{\:pq_j\:>\:0\:\right\}}\lambda({\rm d}y)\frac{\left|w_j(y)p(y)\right|^2}{q_j(y)\sigma_{ij}(x,y)}\left|\frac{f(y)}{p(y)}-\lambda f\right|^2\\&\;\;\;\;\;\;\;\;+2\cdot 1_{\left\{\:pq_i\:>\:0\:\right\}}(x)\frac{w_i(x)p(x)}{q_ix)}\left|\frac{f(x)}{p(x)}-\lambda f\right|^2\sum_{j\in I}\int\lambda({\rm d}y)\frac{w_j(y)p(y)}{\sigma_{ij}(x,y)}\end{split}\tag2\end{equation} for all $i\in I$, $x\in E$ and $w\in L^2(\mu)^I$.

How can we apply the Lagrange multiplier rule and determine the minimizer $w$?

My first problem is that I don't know how I need to incorporate the equality constraint given by the definition of $C$.


  • $I$ is a finite set;
  • $p,q_i$ are probability densities on a measure space $(E,\mathcal E,\lambda)$;
  • $\mu:=p\lambda$;
  • $f\in\mathcal L^2(\lambda)$ with $\{p=0\}\subseteq\{f=0\}$;
  • $\lambda f:=\int f\:{\rm d}\lambda$;
  • $w_i:E\to\mathbb R$ is $\mathcal E$-measurable with $\{q_i=0\}\subseteq\{w_ip=0\}$ for $i\in I$ with $\{pf\ne0\}\subseteq\left\{\sum_{i\in I}w_i=1\right\}$;
  • $\sigma_{ij}:E^2\to\mathbb R$ is $\mathcal E^{\otimes2}$-measurable with $\sigma_{ij}(x,y)=\sigma_{ji}(y,x)$ for all $(i,x),(j,y)\in I\times E$ and $\sum_{j\in I}\int\lambda({\rm d}y)w_i(x)q_j(y)\sigma_{ij}(x,y)=1$ for all $(i,x)\in I\times E$.

Remark: I'm actually searching for $\mathcal E$-measurable $w_i:E\to\mathbb R$ with $\{q_i=0\}\subseteq\{w_ip=0\}$ for all $i\in I$, $\{pf\ne0\}\subseteq\left\{\sum_{i\in I}w_i=1\right\}$ and minimizing $F(w)$. I guess the description above is the best way to formulate this as an optimization problem, but please let me know if you think that I should search for $w$ in a different Banach space or use a different set $C$ (maybe incorporating some of the other requirements mentioned before).


If $w$ is a minimizer then it holds $$ F'(w)h = 0 $$ for all $h$ such that $\sum_{i\in I}h_i=0$ (i.e. for all $h=u-w$ with $u\in C$). To see this, fix $h$ with $\sum_{i\in I}h_i=0$. Then $w+th$ is in $C$ for all $t$, and $F(w+th)-F(w)\ge0$. Dividing by $t$ and passing to the limits $\searrow0$ and $t\nearrow0$ gives the claim.

Define the mapping $T: (L^2)^I \to L^2$ by $$ Th = \sum_{i\in I}h_i. $$ Now the optimality condition above implies $F'(w) \in N(T)^\perp$ (where $N$ is null-space). By the closed range theorem, $F'(w) \in R(T^*)$, i.e., there is $z\in L^2$ such that $F'(w) = T^*z$ or equivalently $$ F'(w) h = \sum_i (h_i ,z), $$ or $$ F'(w)_i = z, $$ where $F'(w)_i$ is such that $ F'(w)h = \sum_{i\in I} F'(w)_ih_i$.

  • $\begingroup$ (a) Am I missing something or doesn't ${\rm D}F(w)h=0$ hold for all $h\in L^2(\mu)^I$ (not only those satisfying $\sum_{i\in I}h_i=0$)? (b) The rest is clear to me except what exactly "$F'(w)_i=z$" means. I guess you're identifying $\mathfrak L(L^2(\mu)^I,\mathbb R)\cong L^2(\mu)^I$, but what is ${\rm D}F(w)$ under this identification precisely? (c) How can we proceed to find a closed form expression of a minimizer? $\endgroup$ – 0xbadf00d Oct 25 at 17:36
  • $\begingroup$ Regarding (b): Added the identification in the question (Hopefully didn't make any mistake. $\endgroup$ – 0xbadf00d Oct 25 at 18:11
  • $\begingroup$ (a) no, see edit, (b) I use this identification, also see edit, (c) no idea. Is the functional convex? It seems to be cubic in $w$. $\endgroup$ – daw Oct 25 at 20:12
  • $\begingroup$ (b) So, you use the same identification as I am, right? (c) You mean the functional $F$? Not sure. What can we infer from $({\rm D}F(w))_i=z$ for all $i\in I$? For example, what could we do in the case $I=\left\{1\right\}$ where we trivially know that the solution is $w_1=1$. $\endgroup$ – 0xbadf00d Oct 26 at 8:14
  • $\begingroup$ (d) Couldn't we minimize the integrand in the definition of $F(w)$ pointwise? This would yield this problem: math.stackexchange.com/q/3409501/47771. $\endgroup$ – 0xbadf00d Oct 27 at 7:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.