3
$\begingroup$

While studying calculus of variations I came across the following problem: $$ \min_{\substack{x\in X \\ y\in Y}} F(x,y) \text{ s.t. } L(x,y) =0 $$ where $F$ is a convex functional, $L$ is a differential operator and $X$, $Y$ are Banach spaces. When, for each $x$, there is a unique $y$ such that $L(x,y)=0$ we define the operator $\phi(x) = y$ so we can study the reduced problem: $$ \min_{x\in X} \tilde{F}(x) = \min_{x\in X} F(x,\phi(x)) $$

The necessary and sufficient condition for optimality is $$ \langle {\tilde {F}}^{\,\prime}(x) \,, x-\tilde{x}\rangle_{X^*\,X} \geq 0,\quad \forall \tilde x \in X. $$ The Euler-Lagrange equation is $\tilde F'(x) = 0$. I cannot convince myself that the Euler-Lagrange equation is the same as the KKTs for the full problem involving the adjoint: \begin{align*} \mathcal{L}_x(x,y,\lambda)&=0 \\ \mathcal{L}_y(x,y,\lambda)&=0 \\ \mathcal{L}_\lambda(x,y,\lambda)&=0 \end{align*}

where $\mathcal{L}(x,y,\lambda) = F(x,y) - \lambda L(x,y)$.

Can you please help me see the connection?

$\endgroup$
  • $\begingroup$ see the Athoch's book [Variational Analysis in Sobolev and BV Spaces: Applications to PDEs and Optimization.] ( amazon.com/Variational-Analysis-Sobolev-Spaces-Applications/dp/…) $\endgroup$ – MathOverview Aug 21 '16 at 11:42
  • $\begingroup$ Which explicit manner for $\tilde{F}$? $\endgroup$ – MathOverview Aug 21 '16 at 12:03
  • $\begingroup$ I have made this explicit now: $\tilde F(x) = F(x, \phi(x))$. $\endgroup$ – stephn28 Aug 21 '16 at 12:10
  • $\begingroup$ The minimizatin will be $\min_{x\in X} \tilde{F}(x) = \min_{x\in X} F(x,\phi(x))?$ $\endgroup$ – MathOverview Aug 21 '16 at 12:13
  • $\begingroup$ Yes, sorry for that $\endgroup$ – stephn28 Aug 21 '16 at 12:16
2
$\begingroup$

Here is my hand wavy answer. I was wondering about the same thing and I think I arrived at the answer from chapters 7 and 9 in 1. This could all be easily formalised of course.

Suppose we are trying to minimize a function. The Euler-Lagrange(E-L) optimality conditions are derived by stating that perturbing a function at its optimum point in all feasible directions would not yield a smaller minimum. Therefore the Frechet differential in directions of its tangent space is zero. Said in another way, by moving along vectors in the tangent cone of the constraint set the function value would only increase or remain the same.

The KKT conditions is saying a similar story, but is like a dual version. It says that the negative gradient or direction of decrease of the function lies in the normal cone which is the dual of the tangent cone. Therefore the direction of decrease of the function has all its components perpendicular to the constraint set at the optimum point or lies in the normal cone. Or said in another way by moving along vectors in the tangent cone we would only increase the function or keep it the same just like in the E-L optimality conditions.

The E-L equation is stated in terms of directional derivatives. The KKT conditions are stated in terms of subdifferentials.

The figure shows the normal cone in green and shows in red (very badly drawn by me) the tangent cone directions. The Euler-Lagrange conditions are formulated in this tangent cone. The KKT in the normal cone

The figure shows the normal cone in green and shows in red (very badly added by me) the tangent cone directions. The Euler-Lagrange conditions are formulated in this tangent cone. The KKT in the normal cone.

Luenberger, D.G., Optimization by vector space methods (Series in Decision and Control), New York-London. Sydney-Toronto: John Wiley and Sons, Inc., XIII, 326 p. (1969). ZBL0176.12701.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.