I have a data set. A matrix $X$, $1300 \times 20$ and output vector $\mathbf{y} \in \Bbb R^{20}$ $$\mathbf{y} = \begin{bmatrix} 100\\100\\\vdots\\100\end{bmatrix}$$ I am trying to run OLS on this data with $\boldsymbol{\beta} \geq \mathbf{0}$ and additional inequality constraints.

I first attempted to solve without the inequality constraints with a Python function that uses coordinate descent. This converged and this test proved I could find a global minimum. But the code doesn't accept constraints beyond $\boldsymbol{\beta} \geq \mathbf{0}$, so it's not enough for my purposes.

I then tried another Python function that uses sequential quadratic programming (SQP), which didn't converge to the global minimum. It should produce the same result since it's the same problem, but it didn't. You can see the details of my code here:

how to stop fmin_slqsp from converging to local minimum?

I determined it's likely the SQP method reached a local minimum whereas the coordinate descent method didn't.

Why would sequential quadratic programming fail to find global minimum?

  • $\begingroup$ Why don't you use CVXPY? $\endgroup$ – Rodrigo de Azevedo Mar 8 at 18:03
  • $\begingroup$ @RodrigodeAzevedo That looks promising. Do you have any recommendations for which methods specifically I should look at that would be useful for my case? $\endgroup$ – Stan Shunpike Mar 8 at 18:08
  • $\begingroup$ That is the whole point of CVXPY. You don't worry about methods. You just write up your optimization problem and CVXPY takes it from there. If the problem is not convex, CVXPY will complain. Good for prototyping. $\endgroup$ – Rodrigo de Azevedo Mar 8 at 18:13
  • 1
    $\begingroup$ It works much better but the solution is still off by a bit. It's 1444.482627745685. $\endgroup$ – Stan Shunpike Mar 8 at 22:01
  • 1
    $\begingroup$ @RodrigodeAzevedo Yes, this worked. This solved the issue. Fantastic, thank you so much for your help. $\endgroup$ – Stan Shunpike Mar 9 at 1:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.