Maximum of Lagrange function I am trying to understand if the following holds
$$\max_{\lambda \succeq 0} L(x, \lambda) =\max_{\lambda \succeq 0} \left( f_0(x) + \sum_{i=1}^m \lambda_i f_i (x)\right)  =\begin{cases}
f_0(x), \quad f_i(x) \leq 0, \forall i \\
\infty,\quad \text{otherwise}.
\end{cases}\tag{1}$$ From $(1)$ we have
$$ \max_{\lambda \succeq 0} \left( f_0(x) + \sum_{i=1}^m \lambda_i f_i (x)\right) = f_0(x) + \max_{\lambda \succeq 0} \left(\sum_{i=1}^m \lambda_i f_i (x)\right).\tag{2}$$ Could you please some one explain analytically (mathematically) how we get $(1)$? Shall we prove that
$$\max_{\lambda \succeq 0} \left(\sum_{i=1}^m \lambda_i f_i (x)\right)=\begin{cases}
0, \quad f_i(x) \leq 0, \forall i \\
\infty,\quad \text{otherwise}.
\end{cases}$$ holds?
 A: Suppose that $f_{i_0}(x) > 0$ for some $i_0$. Then
$$\max_{\lambda \succeq 0} \left( f_0(x) + \sum_{i=1}^m \lambda_i f_i (x)\right) \ge f_0(x) + \sum_{i=1}^m \lambda_i^* f_i (x)$$
where $\lambda_i^* = 0$ for all $i \ne i_0$ and $\lambda_{i_0}^* = u > 0$. Hence
$$\max_{\lambda \succeq 0} \left( f_0(x) + \sum_{i=1}^m \lambda_i f_i (x)\right) \ge f_0(x) + \sum_{i=1}^m \lambda_i^* f_i (x) = f_0(x) + uf_{i_0}(x)$$ for all $u>0$. As  $f_{i_0}(x) > 0$  RHS may be bigger then any number $M>0$ (when $u$ is big). Hence LHS may be bigger then any number $M>0$ (when $u$ is big). But LHS doesn't depend on $u$. Hence LHS is equal to $+\infty$.
Now suppose that $f_i(x) \le 0$ for all $i$.
Then $f_0(x) + \sum_{i=1}^m \lambda_i f_i (x) \le f_0(x) + \sum_{i=1}^m 0$ for all $\lambda \ge 0$ and hence $\max_{\lambda \succeq 0} \left( f_0(x) + \sum_{i=1}^m \lambda_i f_i (x)\right) \le f_0(x)$.
But
$f_0(x) + \sum_{i=1}^m \lambda_i f_i (x) = f_0(x)$ when $\lambda = 0$.
Hence $\max_{\lambda \succeq 0} \left( f_0(x) + \sum_{i=1}^m \lambda_i f_i (x)\right) \ge f_0(x)$.
We got that $\max_{\lambda \succeq 0} \left( f_0(x) + \sum_{i=1}^m \lambda_i f_i (x)\right) \le f_0(x)$ and $\max_{\lambda \succeq 0} \left( f_0(x) + \sum_{i=1}^m \lambda_i f_i (x)\right) \ge f_0(x)$. Thus $\max_{\lambda \succeq 0} \left( f_0(x) + \sum_{i=1}^m \lambda_i f_i (x)\right) = f_0(x)$.
Addition (answer to question from comments):
Put $A = \max_{\lambda \succeq 0} \left( f_0(x) + \sum_{i=1}^m \lambda_i f_i (x)\right)$. We have: $A \ge f_0(x) + uf_{i_0}(x)$ for all $u>0$. Put $B=f_0(x)$, $C = f_{i_0}(x)$. We have $A \ge B + Cu$ for all $u > 0$, where $A, B, C$ don't depend on $u$ and where $C > 0$. We know that $A \ge B+C$ (because we may put $u=1$), we know that $A \ge B+2C$, we know that $A \ge B+10000C$ and so on. Hence $A = +\infty$.  If $u$ is not too big then we still have $A \ge B + Cu$, this inequality works but for small $u$ it's not interesting.
