I've often seen Lagrange Multipliers explained intuitively as follows:

If we are trying to optimize $f(x,y) = z$ subject to the constraint $g(x,y) = 0$, then we are looking for those points in which the gradients of both functions are parallel. This is often described using graphical arguments about the contour lines of $f$ being parallel to $g$.

My issue with this is that my mind automatically starts thinking up a counter example in which $f$ and $g$ are perpendicular at the maximum point of $f$ subject to $g$. I haven't been able to actually come up with a suitable pair of functions that has this property, though.

It is fairly clear that the "standard" intuition for the concept of Lagrange Multipliers isn't a rigorous proof, and I know that there are complete proofs that handle this case quite well. However, I'd still like to be able to explain Lagrange Multipliers using an intuitive argument. Is there such an argument I might be able to make that doesn't have such a glaring counter-example? Alternately, is my counter-example even possible, or would it mean that we can't optimize $f$ subject to $g$ anyway?

  • 1
    $\begingroup$ I think not the gradients of both functions, but the gradient of the contour curve of the objective function must be parallel to the gradient of the constraint function. In fact, the local optimum occurs when the two coincide. Intuitively, if the two gradients are perpendicular (and the contour curve intersects the contraint curve), then the contour curve (consequently the value of the objective function) can be raised up so that it will still intersect the constraint curve. $\endgroup$ – farruhota Jun 1 '17 at 14:42

You cannot construct such a counterexample. If the contour lines are perpendicular at $(x,y)$, you have $$\nabla f(x,y)^\top \nabla g(x,y) = 0.$$ In particular, $\nabla f(x,y)$ and $\nabla g(x,y)$ are linearly independent.

Now, consider $$e(x,y,r) = (f(x,y) - r, g(x,y)).$$ You have $e(x,y,0) = (f(x,y),0)$. Moreover, the Jacobian of $f$ w.r.t. $(x,y)$ is invertible. Hence, the implicit function theorem gives you continuous curves $x(\cdot)$ and $y(\cdot)$, such that $$e(x(r), y(r), r) = (f(x,y) - r, 0)$$ holds for $|r|$ small enough. Hence, $(x,y)$ cannot be a local maximizer/minimizer.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.