5
$\begingroup$

Why does this method actually work?I learnt that a while ago and have been using it ever since. I apply it to almost every optimisation problem. This method just seemed too good to be true. Now, I wonder why this works. (The answer does not need to be rigorous, an intuitive sketch will work just fine for me.)

$\endgroup$
2
  • $\begingroup$ See also math.stackexchange.com/a/815613/50739 $\endgroup$
    – arkeet
    Sep 29, 2016 at 18:56
  • $\begingroup$ @arkeet I did not understand the answer to that post :( $\endgroup$
    – user369582
    Sep 29, 2016 at 19:00

1 Answer 1

13
$\begingroup$

In the simplest case, there is a constraint curve $C:g(x,y)=0$ in the $xy$-plane. Above it is a surface $z=f(x,y)$. If all the points on $C$ are plugged into $f$, then you get some sort of path wondering around the hills and valleys of the surface. In a Lagrange problem, you want to find the highest (or lowest) elevation on that path.

If you look down at the $xy$-plane, you can see $C$ and also a bunch of concentric(-ish) level curves $k=f(x,y)$. Now you want find the level curve of $f$ with the largest $k$ that intersects $C$. So pick out a level curve and notice that it intersects $C$ a couple times. Now start increasing $k$. At some point, there will be a very last level curve that intersects $C$. In my mind the level curves are shrinking with increasing $k$ until that last curve is just touching $C$ and any larger $k$ gives a curve that misses $C$.

If these curves are smooth (the level curves and $C$) then the last time they touch, they are tangent to each other. Which means their slopes in the $xy$-plane are the same. Which means their normal vectors are parallel. Which means the gradients of the 2-variable parent functions are parallel. So we write $\nabla f = \lambda \nabla g$.

If this doesn't explain it well enough, try imagining me waving my hands about in an instructive fashion.

$\endgroup$
2
  • $\begingroup$ @BGoddard One thing i don't get while deriving Lagrange's multiplier is why do we assume f takes local maximum value when it is tangential to g? Can't there be a case wherein the contour line of f which intersects g is not tangential to g? $\endgroup$ Oct 10, 2017 at 17:45
  • $\begingroup$ @krazykode101 We don't assume $f$ takes a local maximum tangent to $g$, we assume the new curve given by $g(x,y)=0$ and $z=f(x,y)$ takes a local maximum. In regards to your second question, there are many such cases where the contour of $f$ intersects $g$ and they are not tangent, we just don't care about them since they don't represent an extreme point. $\endgroup$
    – Vityou
    Mar 2, 2023 at 18:49

You must log in to answer this question.