Skip to main content
8 votes

Lagrangian function (where does it come from?)

Here's an explanation to build intuition about Lagrange multipliers. Suppose you need to optimize (maximize or minimize) $f$ under the constraint that $g(x)=0$. Note that if your constraint is of the ...
Stefan Lafon's user avatar
  • 12.4k
5 votes

Why the Lagrangian function is has different sign depending on the book?

In the case of an equality constraint, it is not important, though one might prefer to write $\mathcal{L}$ and $g$ in such a way that the Lagrange multiplier has a particular meaning. For example, in ...
smcc's user avatar
  • 5,714
5 votes
Accepted

Maximising $x+y$ such that $x^4=(x-1)(y^3-23)-1$ using Lagrange Multipliers

Remark: First, this problem cannot be solved with Lagrange Multiplier technique as $(x,y)\in \mathbb{N}_+^2$. We have: $$(x-1)(y^3-23)= x^4 +1 =(x^4-2x^2+1)+2x^2=(x^2-1)^2 + 2x^2 = (x-1)^2(x+1)^2 +2x^...
NN2's user avatar
  • 16.1k
4 votes
Accepted

Find the maximum and minimum of a function with three variables

Since $\lim_{\|(x,y,z)\|\to\infty}f(x,y,z)=0$, the function $f$ has to have a maximum at some point, and then $\nabla f$ will have to be $(0,0,0)$ at that point. If you solve the system $\nabla f(x,y,...
José Carlos Santos's user avatar
4 votes
Accepted

How to prove there's no global maxima?

If $y=x^2+2$, then $x^2+y^2=x^2+(x^2+2)^2=x^4+5x^2+4$. But then, if$$\varphi(x)=f(x,x^2+2)=x^4+5x^2+4,$$then $\varphi|_{[0,\infty)}$ is a strictly increasing function, whereas $\varphi|_{(-\infty,0]}$ ...
José Carlos Santos's user avatar
3 votes

Least Squares with Inequality Constraints

You have two sets of constraints: $x_1 + x_2 + x_3 = 1$ $x_i \in [0,1]\;\forall i$ In matrix notation you'd get something like this: $\min \;\;(Ax-b)^T(Ax-b)$ $s.t.$ $1^Tx=1$ $x \geq 0$ $x \leq 1$ So ...
Annika's user avatar
  • 6,928
3 votes
Accepted

Lagrange Multipliers Question - some extremum points are missed (not detected) by the method

The method of Lagrange multipliers works when we are after the locally extreme points of a class $C^1$ function $f$ on a region of the type $g(x_1,x_2,\ldots,x_n)=k$, for a class $C^1$ function $g$. ...
José Carlos Santos's user avatar
3 votes

Lagrange Multipliers Question - some extremum points are missed (not detected) by the method

This is an additional remark to the correct reply from José: it is possible (though not always feasible) to introduce new variables and in the process shift the problem to a higher dimensional one ...
maxmilgram's user avatar
  • 3,745
3 votes

Finding the largest first entry of points in a subset of $\mathbb{R}^n$

$$x_1^2=(x_2+\cdots+x_n)^2\leq (x_2^2+\cdots+x_n^2)(n-1)=(1-x_1^2)(n-1)$$ by Schwarz inequality. The bound $x_1\leq \sqrt{\frac{n-1}{n}}$ is reached by taking $x_2=\ldots=x_n=-\frac{1}{\sqrt{n(n-1)}}.$...
Gérard Letac's user avatar
3 votes
Accepted

Lagrangian function (where does it come from?)

I’ll try to stick with your notation. You are minimizing $f(x,y)$ subject to the constraint that $g(x,y) = c$. You can learn about Lagrange multipliers without mentioning the Lagrangian. Once we write ...
littleO's user avatar
  • 52.2k
3 votes
Accepted

Gaussian mixture model with penalty

From $$ \frac{\partial}{\partial \pi_m} \left[ \sum_{i=1}^n \sum_{m=1}^M h_{im} \left[\log (\hat{\pi}^{(0)}_m)+\frac{1}{\hat{\pi}^{(0)}_m}\left(\pi_m-\hat{\pi}^{(0)}_m\right)\right] - n\lambda D_{f} \...
Cesareo's user avatar
  • 33.8k
3 votes
Accepted

Advice on solving Lagrange Multiplier equations

From $$ \cases{ \frac{1}{a^2bc} = 18\lambda a \\ \frac{1}{ab^2c} = 2\lambda b \\ \frac{1}{abc^2} = 2\lambda c }\Rightarrow\cases{ \frac{1}{abc} = 18\lambda a^2 \\ \frac{1}{abc} = 2\lambda b^2 \\ \...
Cesareo's user avatar
  • 33.8k
3 votes

How to prove there's no global maxima?

Geometrically, $f(x,y)=x^2+y^2$ is the squared distance of a point from the origin. $y-x^2=2 \implies y=x^2+2$ is a parabola with vertex at $(0,2)$. Since the coefficient of the quadratic term is ...
TurlocTheRed's user avatar
  • 5,979
3 votes
Accepted

An exercise question in the probability theory class

Let's first of all allow some of the $C$s to be equal. The derivative of the variance w.r.t. $C_i$ is $\frac2K(C_i-m)$ where $m$ is the expectation of $X$ = the mean of the $C$s. Therefore, pushing ...
Gareth McCaughan's user avatar
3 votes
Accepted

Show that minimum of $f(x,y,z) = x^2 + y^2 + z^2$ subject to constraint $x^4 + y^4 + z^4 = 1$ is $1$

$x = 2\lambda x^3$ Either $x = 0$ or $\lambda =\frac {1}{2x}$. You have investigated what happens if $\lambda = \frac 1{2x}$ but what about $x = 0?$
user317176's user avatar
  • 11.5k
2 votes

Find the minimum and maximum values of the function $f(x, y)=x^2y+x+y$ subject to the constraint $𝑥𝑦=4$.

There is no need to use Lagrange multipliers, as you can use the constraint to find $x$ or $y$ in terms of the other, so you can reduce it to a single variable function. If $xy=4$, then $y=4/x$ and $f(...
H. sapiens rex's user avatar
2 votes
Accepted

Why is this expression the Lagrangian dual problem of this QCQP?

Maybe I can write a step by step explanation to clarify what I think are your confusions. Let $p^*$ denote the optimal value of the original problem and $F$ its feasible set. We start with that ...
Michal Adamaszek's user avatar
2 votes

Why is this expression the Lagrangian dual problem of this QCQP?

Because from $$ \nabla_xL=C_0 x+\sum_i\lambda_i A_i x = 0 $$ pre multiplying by $x'$ we get $$ x'C_0 x + \sum_i\lambda_i x'A_ix = x'C_0 x + \sum_i\lambda_i b_i=0 $$ then $x'C_0 x = -b'\lambda$ and $\...
Cesareo's user avatar
  • 33.8k
2 votes

How to maximize $\min_k a_k$ under constraints $\sum_k a_k=A$ and $\sum_k a_k^2=B$?

In the case $n=3$ the command of Mathematica 13.3 (Capital letters as parameters and subscripts are not welcome in MMA.) ...
user64494's user avatar
  • 5,831
2 votes

If $a,b,c\ge 0: ab+bc+ca=1,$ prove $\frac{1}{\sqrt{a+1}}+\frac{1}{\sqrt{b+1}}+\frac{1}{\sqrt{c+1}}\le 1+\sqrt{2}.$

If $c=0$, so $ab=1$ and by C-S $$\sum_{cyc}\frac{1}{\sqrt{1+a}}=\frac{1}{\sqrt{1+a}}+\frac{1}{\sqrt{1+b}}+1\leq\sqrt{2\left(\frac{1}{1+a}+\frac{1}{1+b}\right)}+1=$$ $$=\sqrt{\frac{2(2+a+b)}{2+a+b}}+1=\...
Michael Rozenberg's user avatar
2 votes

Analytical solution for equality-constrained QCLP

First reduce the order of the system by 1, by solving the linear system $ \omega^T \mathbf{1} = 1 $ This will give $ \omega = \omega_0 + M u $ where $u \in \mathbb{R}^{n-1} $ and $ M \in \mathbb{R}^{n ...
c'est pas normale's user avatar
2 votes

If $xy+yz+zx+xyz=4,$ prove $\frac{1}{\sqrt{x+1}}+\frac{1}{\sqrt{y+1}}+\frac{1}{\sqrt{z+1}} \leq 1+\frac{2}{\sqrt{3}}.$

For $z=0$ we obtain $xy=4$ and we need to prove that: $$\frac{1}{\sqrt{x+1}}+\frac{1}{\sqrt{y+1}}\leq\frac{2}{\sqrt3}$$ or $$\frac{1}{x+1}+\frac{1}{y+1}+\frac{2}{\sqrt{(x+1)(y+1)}}\leq\frac{4}{3}$$ or ...
Michael Rozenberg's user avatar
2 votes

Finding $\small{\max\limits_{a+b+c=3}\frac{1}{\sqrt{a^2-3a+3}}+\frac{1}{\sqrt{b^2-3b+3}}+\frac{1}{\sqrt{c^2-3c+3}}.}$

For $a=b=c=1$ we obtain a value $3$. We'll prove that it's a maximal value. Indeed, by C-S $$\sum_{cyc}\frac{1}{\sqrt{a^2-3a+3}}\leq\sqrt{\sum_{cyc}\frac{1}{(a^2-3a+3)(2a+3)}\sum_{cyc}(2a+3)}$$ and it ...
Michael Rozenberg's user avatar
2 votes

Finding $\small{\max\limits_{a+b+c=3}\frac{1}{\sqrt{a^2-3a+3}}+\frac{1}{\sqrt{b^2-3b+3}}+\frac{1}{\sqrt{c^2-3c+3}}.}$

Suppose $f(x,y,z)=x+y+z-C=0$ And $g(x,y,z)=\mu(x)+\mu(y)+\mu(z)$ By Lagrange Multipliers, we have: $\lambda=d\mu/dx=d\mu/dy=d\mu/dz$ $\mu(x) = \frac{1}{\sqrt{x^2-3x+3}}$ $\frac{d \mu}{dx}=\frac{(2x-3)...
TurlocTheRed's user avatar
  • 5,979
2 votes
Accepted

Show that Lagrange multiplier method fails to solve $\min(x^2+y^2)$ subject to $(x-1)^3-y^2=0$

First, the method of Lagrange does not apply, it is not that it fails. The gradient of the constraint is zero at the solution. The constraint qualification for the method to apply is that the active ...
copper.hat's user avatar
  • 173k
2 votes
Accepted

What is the Lagrange Multiplier value?

Yes, indeed. Nice observation. From $\nabla f = \lambda \nabla g$, taking the scalar product with $\nabla g$ follows the equation for $\lambda$ which you wrote.
Giuseppe Negro's user avatar
2 votes

Linearizing SOS1

Given that $\lambda \ge 0,$ $$\lambda = 0 \vee l - x = 0$$ can be linearized as $$\lambda \le M z$$ $$L (1-z) \le l - x \le U (1-z)$$ where $z$ is a new binary variable and $M, L, U$ are suitably ...
prubin's user avatar
  • 5,168
2 votes
Accepted

Choose the $w_i>0$ such that $\sum_{i=1}^dw_ia_i$ is minimized and $\sum_{i=1}^dw_i=1$

If we find the smallest $a_i$ and set that $w_i$'s weight to 1, that should satisfy the constraint and minimize $\sum_{i=1}^dw_ia_i$. There is no reason to give weight to any $a_i$ than the minimal ...
FallenNugget's user avatar
2 votes

Complicated Inequality Proof, Variables Subject To Constraint

Too long for comment: Rewrite the inequality as: $$ 5\sum_{cyc}\frac{a^2bc}{a^2+b^3+c^3}\leq 1, $$ By AM-GM we have $$a^2bc \leq \frac{(1+a)^4}{256},$$ and by the power-means inequality we have $$ \...
V.S.e.H.'s user avatar
  • 2,739
2 votes
Accepted

Finding Extremes of Multivariable Function $f(x,y,z)=xz−yz $

The Lagrange multiplier condition is that to find constrained extrema of $f$ subject to $g_1, g_2 = 0$, we solve the system $\nabla f = \lambda g_1 + \mu g_2$ (or some equivalent condition). In this ...
Osama Ghani's user avatar
  • 2,882

Only top scored, non community-wiki answers of a minimum length are eligible