# Constrained Optimization: Lagrange Multipliers

Let there be a function $f(x)$ that we want to find a local maximum, so expressing the problem as below \begin{align} \max\:\, f(x),& \enspace x \in R^{n} \\ \text{s.t.} \quad g_{1}(x) &= b_{1} \\ \quad \quad \quad &\vdots \\ g_{M}(x) &= b_{M} \\ \nabla f(x) &= \sum_{i=1}^{M} \lambda_{i}\nabla g_{i}(x) \end{align}

Why do we represent $\nabla f(x)$ as a linear combination of constraint's gradients when using lagrange multipliers? I do get sort of an intuition, but I'd like an explanation to fully get it, please.

People constantly use this when maximizing utility in microeconomics when individuals are subject to some sort of restriction and that's why I'd like to know more.

First, note that (when non-zero,) $\nabla f$ is a vector that indicates in each point the direction of greatest change for $f$. It is orthogonal to the $n-1$ directions that are tangent to the local level surface of $f$.

The same goes for $\nabla g_i$: it indicates the direction of greatest change for $g_i$. Now you want to consider only some specific level surfaces of the $g_i$ (the $M$ constraints). Therefore, if $x$ is a point where the constraints are met, you may move only in the directions that are tangent to the level surfaces of all $g_i$'s (otherwise you are going to leave them).

Observe that:

Moving tangentially to all $g_i$'s level surfaces means moving perpendicularly to all of the $\nabla g_i$.

Now at a local extremum, the directions tangent to all $g_i$ level surfaces have to be tangential to the level surface of $f$ as well (otherwise you could increase or decrease $f$ at will while keeping the constraints satisfied).

In other words, at such a point: "if a direction is authorised, then it is perpendicular to $\nabla f$".

Rephrasing again: if a direction "is perpendicular to all $\nabla g_i$, then it is perpendicular to $\nabla f$".

This last sentence is equivalent to "$\nabla f$ lies in the subspace spanned by the $\nabla g_i$". Which is saying that there are $\lambda_i$ such that...