I've a topic in my notes "The method of Lagrange's multipliers" which is described as follows:

Let $U$ be an open set in $\mathbb R^n$.Let $f\in C^1(U,\mathbb R)$ and let $$g_i(x)=0,~~~~~~~~i=1,\ldots,m$$
be a collection of equality constraints with $m\lt n$.Now consider the system of non-linear equations$$f(x)=a$$ $$g_i(x)=0,~~~~i=1,\ldots,m$$
$x_0\in U$ is a local maximum if $f(x_0\geq f(x)$ for all $x\in U$ near $x_0$ which also satisfies the above constrints.A local minimum is defined similarly.

Let $F:U\times \mathbb R\to \mathbb R^{m+1}$ be defined by

$$F(x,a)=\left[ \begin{array}{ccc} f(x)-a \\ g_1(x)\\ \vdots \\ g_m(x)\end{array} \right] $$

Now consider the $m+1\times n$ Jacobian matrix ,the matrix of linear transformation , $F'(x,a)$ w.rt. usual basis for $\mathbb R^n$ and $\mathbb R^{m+1}$ at $x_0$ :

$$\left[ \begin{array}{ccc} \partial_1 f(x_0) \ldots \partial_n f(x_0) \\ \partial_1g_1(x_0)\ldots \partial_n g_1(x_0)\\ \vdots ~~~~~~~~~~~~\vdots\\\partial_1 g_m(x_0) \ldots\partial_n g_1(x_0)\end{array} \right] $$

If this matrix has rank $m+1$ then some $m+1\times m+1$ sub matrix has non-zero determinant.It follows from the implicit function theorem that there exist $m+1$ variables $x_{i_1},\ldots,x_{i_{m+1}}$ such that the system $$F(x,a)=0$$
specifies these $m+1$ variables as a function of remaining $n-(m+1)$ variables and $a$ in an open set of $\mathbb R^{n-m}$ .

Thus ,there is a solution $(x,a)$ to equation $F(x,a)=0$ for some $x$ close $x_0$ whenever $a$ is in some open interval .

Therefore , $x_0$ cannot be local maximum or minimum. If $x_0$ is either a local maximum or minimum ,then the above matrix must have rank less than $m+1$ which reqiures the rows to be linearly dependent. Thus ,$\exists m$ scalars $\lambda_1,\ldots,\lambda_m$ and a scalar $\mu$ not all zero s.t.: >$\mu\left[ \begin{array}{ccc} \partial_1f(x_0)\\ \vdots \\ \partial_nf(x_0)\end{array} \right] $ =
$\lambda_1\left[ \begin{array}{ccc} \partial_1g_1(x_0)\\ \vdots \\ \partial_ng_1(x_0)\end{array} \right] $+$\ldots$ +$\lambda_m\left[ \begin{array}{ccc} \partial_1g_m(x_0)\\ \vdots \\ \partial_ng_m(x_0)\end{array} \right] $

I have the following doubts from the above extract:

$1.)$ for what purpose did we define : $$F(x,a)=\left[ \begin{array}{ccc} f(x)-a \\ g_1(x)\\ \vdots \\ g_m(x)\end{array} \right] $$

$2.)$ I can't understand how is it concluded that : " Therefore ,$x_0$ cannot be local maximum or minimum."

Please if anyone can help me with these doubts...


I like your question, thanks for taking the time to write this all out. I'll try to explain as best I can:

1) The definition of $F(x,a)$ is designed so that when $F(x,a) = 0$ we are both satisfying the constraints $g_i$, and aware that $f(x)$ is taking on the value $a$. Essentially, the solution set of $F(x,a) = 0$ is the intersection of the set satisfying the constraints $g_i$ with the level set of $f$. For example, if we consider in $\mathbb{R}^3$

$$g_1 = x^2 + y^2 + z^2 - 4, ~~ f = x + y + z,$$ then $g_1 = 0$ describes a sphere centered at the origin of radius 2 while the level sets of $f$ are planes (specifically, $f = a$ is the plane which intersects each axis at a distance $a$ from the origin). For example, the point $(2,0,0)$ is on the sphere, but $f(2,0,0) \neq 4$ so we should find that the constraint equation is satisfied (a zero in the second entry of $F$), but the first entry will only be satisfied if $a = 2$, since $(2,0,0)$ lies on the level set $f(x) = 2$. And you compute $$F((2,0,0),4) = \left[\begin{matrix} x+y+z - a \\ x^2 + y^2 + z^2 - 4 \end{matrix} \right] = \left[\begin{matrix} -2 \\ 0 \end{matrix} \right]$$ while $$F((2,0,0),2) = \left[\begin{matrix} x+y+z - a \\ x^2 + y^2 + z^2 - 4 \end{matrix} \right] = \left[\begin{matrix} 0 \\ 0 \end{matrix} \right].$$

Our goal is to find an $x_0$ solving $F(x_0,a) = 0$ for the largest (or smallest) possible $a$.

2) The key to understanding this quote is that, for some neighborhood $U$ about $(x_0,a)$, we have a function $\phi:\mathbb{R}^{n-m} \rightarrow \mathbb{R}^{m+1}$ such that $\phi(u) = (x_u,a_u)$ which satisfies $$F(x_u,a_u) = 0 ~ \textrm{ for all } ~u \in U$$ and (because the Jacobian is non-singular) has an open image. Since the image is open we can find some point $(x',a')$ in $\phi(U)$ with $a' > a$.

If this is true, then the level set $f = a'$ also intersects the solution of $g = 0$. This shows $f = a$ is not the maximum value $f$ takes on.

To finish, look again at the example. You already know that the largest value that $f = x + y + z $ can take on for points on the sphere $g_1 = 0$ is $a = 2\sqrt{3}$, and this happens at the point $({2\over\sqrt{3}},{2\over\sqrt{3}},{2\over\sqrt{3}})$ where the plane $f = 2\sqrt{3}$ is tangent to the sphere in the first quadrant. At this point the Jacobian of $F$ breaks down:

$$ \left[ \begin{matrix} 1 & 1 & 1 \\ 2x & 2y & 2z \end{matrix} \right] = \left[ \begin{matrix} 1 & 1 & 1 \\ {4\over \sqrt{3}} & {4\over \sqrt{3}} & {4\over \sqrt{3}} \end{matrix} \right]$$

is definitely rank one. For any $a \in (-2\sqrt{3}, 2\sqrt{3})$, the set of intersection is a circle, the Jacobian of $F(x,a)$ is nonsingular (try one if you like), and you can scoot up to another circle (level set) with a higher value of $f$.

| cite | improve this answer | |
  • $\begingroup$ can you please elaborate me with this: "(because the Jacobian is non-singular) has an open image". $\endgroup$ – spectraa Nov 8 '14 at 2:41
  • 2
    $\begingroup$ This is part of what the implicit function theorem asserts. With nonzero determinant of the Jacobian you get a map from an open set to an open set, such that $\phi$ satisfies $F(\phi(u)) = 0$. Intuitively you can think of the Jacobian as telling you the change in volume between a small section of the domain and its image. See www-math.mit.edu/~djk/18_022/chapter11/section04.html If the matrix is of full rank, it maps a small volume element of $\mathbb{R}^{n-m}$ to a small image with positive $(n-m)$-dimensional volume. $\endgroup$ – Titus Nov 8 '14 at 7:15
  • 2
    $\begingroup$ If the Jacobian mapped a 2-dimensional square to a one-dimensional image, its determinant would be zero. In this case, it's clear that the image will not be open. However, if it is of full rank it will map that tiny square to another tiny parallelogram. Here it's clear that a small 'buffer' about a point in that square will remain about it, and the mapping will be open. $\endgroup$ – Titus Nov 8 '14 at 7:19
  • 2
    $\begingroup$ Perhaps this would make for a good second question if you'd like more detail. "How does a nonsingular Jacobian imply an open image?" Could be a fun discussion... $\endgroup$ – Titus Nov 8 '14 at 7:20
  • 1
    $\begingroup$ If the neighborhood is open, it doesn't include its boundary, so $a$ is not on the boundary and some $a'$ lies above $a$. $\endgroup$ – Titus Nov 9 '14 at 3:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.