1
$\begingroup$

It is widely known that interior point method has been used for convex optimization problem.

However, the intermediary step of interior point method is usually consist of the Newton method. This may make the speed become terribly slow since Newton method is second order method.

Should projected gradient descent be used for solving convex optimization problem instead of interior point method for convex problem ?

$\endgroup$
1
  • 1
    $\begingroup$ There is no one answer here. In general Newton methods are more expensive and faster. $\endgroup$
    – copper.hat
    Jan 2 at 16:53

1 Answer 1

Reset to default
2
$\begingroup$

I think you have it backwards. Newton’s method being second order means that every iteration doubles the digits of accuracy in the solution. This is a good thing! With gradient descent the error decreases only linearly.

There are reasons to avoid Newton’s method in some cases (expense of computing second derivatives, e.g.) but rate of convergence isn’t one of them.

$\endgroup$
1
  • $\begingroup$ You got it right I have been a little bit backward. But is there any case where the computation of second derivative is too expensive and slow that gradient descent may reach the optimal point sooner than newton method $\endgroup$
    – Tuong Nguyen Minh
    Jan 3 at 5:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .