The method of Lagrange multipliers is used to find the extrema of $f(x)$ subject to the constraints $\vec g(x)=0$, where $x=(x_1,\dots,x_n)$ and $\vec g=(g_1,\dots,g_m)$ for $m \leq n$.

Although many textbooks get the final equations by arguing that at an extrema, the variation of $f(x)$ must be orthogonal to the surface $g(x)=0$, the "simpler" approach (and that which is commonly seen in field theory / optimizing functionals) is to construct the Lagrange function $$ L(x,\lambda) = f(x) + \vec\lambda\cdot\vec g(x) $$ and varying w.r.t. $x$ and $\lambda$ to get the vector equations $$ \begin{align} &x:& 0 &= \nabla f(x) + \sum_i \lambda_i \nabla g_i(x) \,, \\ &\vec \lambda:& 0 &= \vec g(x) \ . \end{align} $$

The method only works if the extremal point is a regular point of the constraint surface, i.e. if $\mathrm{rnk}(\nabla\vec g) = m$.

What is the best way of understanding what goes wrong when the extrema is not a regular point of the constraint?

And, most importantly to me, how does this generalize to field theories (i.e. optimizing functionals) with local constraints? What is the equivalent regularity condition for constraints in field theory?

Instructive examples are more than welcome.


Generically, the $m$ equations $g_i(x)=0$ define a manifold $S$ of dimension $d:=n-m$. At each point $p\in S$ the $m$ gradients $\nabla g_i(p)$ are orthogonal to the tangent space $S_p$ of $S$ at $p$. The condition rnk$(\nabla g(p))=m$ means that these $m$ gradients are linearly independent, so that they span the full orthogonal complement $S_p^\perp$ which has dimension $m=n-d$. At a conditionally stationary point $p$ of $f$ the gradient $\nabla f(p)$ is in $S_p^\perp$, and if the rank condition is fulfilled, there will be constants $\lambda_i$ such that $\nabla f(p)=\sum_{i=1}^m \lambda_i\nabla g_i(p)$. In this case the given "recipe" will find the point $p$.

Consider now the following example where the rank condition is violated: The two constraints $$g_1(x,y,z):=x^6-z=0,\qquad g_2(x,y,z):=y^3-z=0$$ define a curve $S\subset{\mathbb R}^3$ with the parametric representation $$S: \quad x\mapsto (x,x^2,x^6)\qquad (-\infty < x <\infty).$$ The function $f(x,y,z):=y$ assumes its minimum on $S$ at the origin $o$. But if we compute the gradients $$\nabla f(o)=(0,1,0), \qquad \nabla g_1(o)=\nabla g_2(o)=(0,0,-1),$$ it turns out that $\nabla f(o)$ is not a linear combination of the $\nabla g_i(o)$. As a consequence Lagrange's method will not bring this conditionally stationary point to the fore.

  • $\begingroup$ Thanks Christian. I never found this explanation completely satisfying - but I think that I'm coming around to liking it now. How does this understanding generalize to functionals - eg optimizing $\int f(x,\phi(x),\phi'(x)) dx$ wrt some constraint $g(\phi(x))=0$? $\endgroup$ – Simon Feb 27 '11 at 21:27
  • 2
    $\begingroup$ A constraint of the form $\int_a^b g(\phi(x))dx=c$ defines a submanifold of codimension one in the infinite-dimensional space of all conceivable functions $\phi(\cdot)$. But now there is no such thing as $\nabla \Phi$ (where $\Phi$ is the "large" integral that has to be minimized), so it is difficult to push the geometric intuition into this setting. In any case, Lagrange's method can be applied also here, but I suggest that you consult a textbook on variational calculus on this matter. $\endgroup$ – Christian Blatter Mar 1 '11 at 9:12
  • $\begingroup$ I was actually looking at a pointwise constraint $g(\phi(x))=0$ - so it's codimension is infinite (I guess). Why is the variational/functional derivative is not a good analog of the partial derivative in the finite dimensional case? $\endgroup$ – Simon Mar 7 '11 at 3:24
  • $\begingroup$ PS sorry it took me so long to reply - you forgot to address your comment to me, so I wasn't notified of it. $\endgroup$ – Simon Mar 7 '11 at 3:26
  • $\begingroup$ If some of the constraints aren't linearly independent then just reduce m to compensate, correct? $\endgroup$ – user10389 Nov 17 '11 at 18:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.