A question about Lagrange multiplier Is there any explanation or interpretation of the concept of Lagrange multipliers $\nabla f(x_0)= \delta \nabla g(x_0) $ for some constant $\delta$ and $f$ is a differentiable function and g is the constraint of $f$. I know that this comes from the proof and I have read the proof, but does it give any geometrical meaning? Also, it is possible that $\nabla f(x_0)= \delta \nabla g(x_0) $ but $x_0$ doesn't give extremal points?
 A: The restriction that $g(x)=0$ means that the allowable differential changes in $x$ are those perpendicular to $\nabla g(x)$. That is, for $x$ moving in the "surface" where $g(x)=0$, $\mathrm{d}g(x)=0$; therefore, $\mathrm{d}x\cdot\nabla g(x)=0$.
So that $f(x)$ is stationary for all allowable differential changes in $x$ (for $x$ moving in the "surface" where $g(x)=0$), we want to have $\nabla f(x)$ parallel to $\nabla g(x)$.  That way, for any $\mathrm{d}x$ so that $\mathrm{d}x\cdot\nabla g(x)=0$ (i.e. $\mathrm{d}g(x)=0$), we also have $\mathrm{d}x\cdot\nabla f(x)=0$ (i.e. $\mathrm{d}f(x)=0$).
When $\nabla f(x)$ is parallel to $\nabla g(x)$, there is a $\delta$ so that $\nabla f(x)=\delta\nabla g(x)$.

However, it is quite possible that $\nabla f(x)=\delta\nabla g(x)$ where $x$ is not an extremal point. Consider
$$
f(x,y,z)=z\text{ and }g(x,y,z)=x^2-y^2+z
$$
where
$$
\nabla f(x,y,z)=(0,0,1)\text{ and }\nabla g(x,y,z)=(2x,-2y,1)
$$
Note that $\nabla f$ is parallel to $\nabla g$ only when $x=y=0$. Since we are constrained to $g(x,y,z)=0$, we get the critical point to be $(0,0,0)$. Note that $f(0,0,0)=0$.

Consider the curves $\gamma_1(t)=(t,0,-t^2)$ and $\gamma_2(t)=(0,t,t^2)$. Both satisfy $g(\gamma_j(t))=0$ and $\gamma_j(0)=(0,0,0)$. However, $f(\gamma_1(t))=-t^2\le0$ and $f(\gamma_2(t))=t^2\ge0$. Thus, $(0,0,0)$ is not extremal.
A: Imagine walking along a trail along the side of a mountain. However, the trail doesn't go all the way to the top, but instead meanders around the hillslope and then comes back down.
You're standing at the highpoint of this trail (which is not the highpoint of the mountain), but you're not satisfied and want to go off-trail to climb higher. From where you're standing, relative to the trail, which direction should you go?
Hopefully a little visualization will convince you that, at the highpoint, the direction of steepest ascent is also exactly perpendicular to the trail.
Now, this relates to the mathematical situation as follows: 


*

*the trail is the contour $g(x,y)=0$

*the direction perpendicular to the trail is the vector $\nabla g$

*the height of the mountain at point $(x,y)$ is $f(x,y)$

*the direction of steepest ascent is the gradient $\nabla f$


Then in the analogy, the equation $\nabla f(x_0,y_0)=\delta \nabla g(x_0,y_0)$ says that the direction of steepest ascent is perpendicular to the trail at the highpoint $(x_0,y_0)$. The scalar $\delta$ is a scaling constant to account for the fact that vectors can be in the same direction but have different lengths.
