# One dimensional constrained optimization: KKT conditions versus irregular points

In a constrained optimization problem, we search local solutions in the regular points that satisfy the KKT solutions as well as the irregular points. All the local solutions are included in these points.

However, in one dimensional problems, e.g.: $$\min_x \{-(x-1)^2 + x \ | \ -1 \leq x \leq 3 \}$$, we search the solution in the KKT system AND at the endpoints $$x = 3, \ x=-1$$. What I don't get is, why are we also looking at the endpoints?

I believe this is because at an active constraint we either have $$x = -1$$ or $$x = 3$$, and both of these constraints have gradients $$\nabla_x (x) = 1$$, and maybe this means these are irregular points? But why are these irregular points? Isn't the singleton $$\{ 1\}$$ linearly independent?

Edit: I know my example is minimizing a concave function, and in such problems the global minimum is on one of the endpoints. But this does not relate to the 'KKT conditions' now.

• My bad. Now realized what is really going on. If we just apply the KKT to the given example, we have the first order condition of the function (without the constraint) given by the case where the Lagrange multipliers are zero. If one of the Lagrange multipliers is non-zero, then the corresponding constraint is active, hence the 'endpoint' solution arises. This now shows a relation between the KKT solution of a single variable model versus the case of taking FOC on the objective and also trying the endpoints.. Commented Apr 23, 2020 at 2:20

For your example problem (P) below \begin{align} \tag{P} \min_x \quad f(x) &:= -(x-1)^2 + x \\ \text{s.t.} \quad g_1(x) &:= -1 - x \leq 0 \\ g_2(x) &:= x-3 \leq 0, \end{align} we form the Lagrangian $$L(x;\lambda_1, \lambda_2) := f(x) + \lambda_1 g_1(x) + \lambda_2 g_2(x)$$ and read off the KKT conditions \begin{align} 1:& \quad \nabla_x L(x^{\star}; \lambda_1, \lambda_2) = 0 = -2(x^{\star}-1) + 1 - \lambda_1 + \lambda_2 \\ 2:& \quad g_1(x^{\star}) \leq 0, \quad g_2(x^{\star}) \leq 0 \\ 3:& \quad \lambda_1^{\star} \geq 0, \quad \lambda_2^{\star} \geq 0 \\ 4:& \quad \lambda_1^{\star} g_1(x^{\star}) = 0, \quad \lambda_2^{\star} g_2(x^{\star}) = 0. \end{align}
By inspection, we can find three KKT points from conditions 1-4: \begin{align} &(x = -3/2, &\lambda_1=0, \quad &\lambda_2=0), \\ &(x = -1, &\lambda_1=5, \quad &\lambda_2=0), \\ &(x = 3, &\lambda_1=0, \quad &\lambda_2=3). \end{align}
As you mention in your comment, the first comes from looking at condition 4 and supposing that $$\lambda_i = 0$$. This then let's us consider the "unconstrained" problem in $$f$$, for which the necessary condition would be $$\nabla_x f(x) = 0$$. It happens at $$x=3/2$$ (which is feasible based on condition 2), so it is a KKT point.
Next we might ask about condition 4 with the $$g_i(x)$$ contributing the zero. This leads to the other edge cases, and the value of the multipliers $$\lambda_1$$ and $$\lambda_2$$ "fall out" based on condition 1.