I am keen to know about the literature landscape for distributed convex optimization methods which use second order information like the Newton step. This is as such a less evolved area compared to first order methods, which have had most of the focus from the research community for large scale optimization (e.g., ADMM etc).

But I see papers like "A distributed newton method for network utility maximization" by Ermin Wei et al. I want to have an understanding of whether these methods have been extended to other areas like Machine Learning and what is the problem size that a state of the art distributed Newton method can handle?

  • $\begingroup$ Have you read the paper? Even if much of it is too technical, it may have a section titled "Discussion" or something similar where it would be likely to answer your questions. $\endgroup$ – Milo Brandt Nov 26 '14 at 0:29
  • $\begingroup$ Yes, I looked at results section and they presented for some small sized networks. So, I was wondering what can be done further. $\endgroup$ – haripkannan Nov 26 '14 at 13:04
  • 1
    $\begingroup$ There's a reason distributed methods so far have favored first-order methods: they're much easier to distribute. The Hessian connects variables to each other much more strongly making the computations harder to distribute. $\endgroup$ – Michael Grant Nov 26 '14 at 14:52

The paper that you mentioned by E. Wei is proposed for network utility maximization (NUM) problems, but recently a new algorithm called Network Newton is proposed that is an approximation of distributed Newton method for solving general distributed convex optimization problems. The authors have also considered the application of algorithm for logistic regression.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.