I am following a book which has a part on numerical optimization techniques. In order to elaborate Karush-Kuhn-Tucker theorem, they gave the following example: enter image description here

When the unconstrained solution $x=A^+ b$ does not lie in the feasible region, they apply an iterative approach to find the solution where the norm of $x$ is equal to $1$. They seem to increase the value of $\lambda$ which causes the optimal solution for $x$ to shrink its norm, until the norm converges to $1$. What I do not understand is that they say they apply gradient ascent to $\lambda$. What is the logic behind applying gradient ascent in this scheme? The aim is to find a solution to $L(x,\lambda)$ where $x$ is optimal and satisfying $x^Tx \leq 1$ but I do not see how applying gradient ascent only to $\lambda$ by calculating $\dfrac{\partial L(x,\lambda)}{\partial \lambda}$and updating the optimal $x$ leads to convergence. What is really going on here?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.