I have been studying the method of Lagrange multipliers for a machine learning course. From the following sources:
I have been able to achieve a core understanding of the method. However this one source:
makes it sound as though I can use comparisons between the slope of the tangent line of the constraint curve $g(x,y,z,..) = 0$ and the slope of the tangent line of the level curve $f(x,y,z..) = d$ at points where these curves intersect to determine which direction along the constraint curve I should move in order to reach a max or min of the surface $f(x,y,z,...)$ subject to the constraint curve $g(x,y,z,..) = 0$.
To quote the source:
"Imagine hiking from left to right on the constraint line. As we gain elevation, we walk through various level curves of f. ...~... .At each level curve, imagine checking its slope — that is, the slope of a tangent line to it — and comparing that to the slope on the constraint where we’re standing.
If our slope is greater than the level curve, we can reach a higher point on the hill if we keep moving right. If our slope is less than the level curve — say, toward the right where our constraint line is declining — we need to move backward to the left to reach a higher point."
Figure 1 in the Wikipedia source seems to give a counter example to this (if I am interpreting things correctly) hence my confusion. It would be a great help if this role of the tangent line in Lagrange Multipliers can be confirmed and the math behind it shown. Moreover, if this is confirmed, I would like clarification on whether the sign of the slopes matters (Which I would think that it does) or if its just the magnitude.
I realize my question is similar to and contains the same subject matter as these questions:
but I'm going to post this question anyway