Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

$$\min(f_0(x))$$ $$\text{s.t. }f_i(x) \le y_i \forall i, i = 1 ,\ldots, m$$

$$f_i : \text{convex};\quad x : \text{variable}$$

It is also considered that $g(y)$ is the optimal value of the problem and $\lambda^*$ is the optimal dual variable.

Then, it is claimed that $$g(z) \ge g(y) - \sum_{i=1}^m \lambda^*_i * (z_i - y_i)\tag{1}$$

Hence $- \lambda^*$ is a subgradient of $g$ at $y$.

Though the material I am reading from (Basic Rules for Subgradient Calculus, slide at 51:40 mins) claims the proof of (1) is straightforward, still I can't figure out how to derive that. Can anybody help ?

My Approach:

Assuming $z_i = f_i(x)$, I get the Lagrangian dual function as $g(z) = f_0(x) + \sum_{i=1}^{m}\lambda_i (z_i-y_i)$. Since $g(y)$ is the optimal value of the problem and $\lambda^*$ is the optimal dual variable, I can write

$$g(y)=f_0(x) + \sum_{i=1}^{m}\lambda^*_i(z_i-y_i)$$ or may be $$g(y)=f_0(x)$$ since $z_i = y_i$. But then I can't figure out how should I use the $\lambda^*$ in the eq (1) which is to be derived.

The unconstrained problem is $$g(y) = \inf_z g(z).$$ Based on problem definition, it is also true that $g(y) \le g(z)$. But how can (1) be derived from these relations? Or, I am doing something wrong assumptions here?

share|cite|improve this question
I'm confused. What is $g(y)$? The value of $f_0$ at its minimum within the admissible region parameterized by $y$, right? If so, what does it mean to equate $g(y)$ with its infimum? – user7530 Dec 31 '12 at 1:34
$g(y)$ is considered as the optimal value of the primal problem. Yes, I think $g$ is the unconstrained optimization problem parameterized with variable $y$, the resource variable in $m$ inequality constraints. Though when I looked at literature, I find that generally the Lagrangian parameters (e.g. $\lambda, \nu$) are considered as dual variables. I am not sure why $y$ is the parameter in this case. Again, since we are trying to find the pointwise infimum of $g(z)$ and $g(y)$ is the optimal value, $y$ should be the minimizer. The Lagrangian part in $g(y)$ is minimized when $z_i = y_i$. – somnathchakrabarti Dec 31 '12 at 1:45
@user7530: For a particular $x$, $f_0(x)$ is the fixed part and unvariant with parameter $y$. That's my understanding from the video lecture slide. What it tells in the lecture is that this problems falls into the category of convex minimization problems with minimal resource allocation, like for $m$ resources corresponding to $m$ inequality constraints. – somnathchakrabarti Dec 31 '12 at 1:52
This is called sensitivity analysis. You will find an answer to your question in almost any book on convex optimization (e.g., First you need to convince yourself that $g$ is a convex function. Secondly (under Slater's qualification condition), $-\lambda^*$ is a subgradient of $g$. – Dominique Jan 30 '13 at 13:36

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.