Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am dealing with nonlinear systems of equations that I am trying to solve numerically. These sets of equations derive from structural mechanics involving strong nonlinearities, like contact. The size of these problems is in the order of ~10 to ~100 degrees of freedom, out of which only a few are subject to nonlinear constraints. All of this currently happens in a path-continuation framework.

I have been trying around solving these equations. I have had acceptable convergence using matlab's fsolve for certain problems and parameters, and I am now switching to python implementations. Python's Scipy Optimization toolbox provides a number of solvers like fsolve and root. Especially root has a range of methods to choose from (hybr,lm,broyden1,etc.), that I am not familiar with. Some converge while others don't.

Do any of you know which method is good for what type of application? Is there any literature that outline the pro's and con's of each method, that I am unable to find?

share|cite|improve this question
up vote 3 down vote accepted

If your system is smooth, I would try to use one of the following solvers: KNITRO, SNOPT, ALGENCAN,IPOPT. The first two are commercial (but they are free for academics with, say, up to 300 variables), the last two are free and open source. They are all optimization solvers, so you could, for example, look for min-norm solution subject to the constraints given by your system.

I would experiment with these solvers from inside a modeling environment like AMPL or GAMS. The benefit is that you write down your model once, and to change solvers you only change one line of code (maybe two, if you start passing options to the solver). I know AMPL very well, and it has a very clear, intuitive language. They have excellent documentation in the form of the AMPL book (now available for free online on the AMPL website). It also implements very good automatic differentiation algorithms, so you don't have to provide Jacobians or Hessians, nor do you have to use finite-difference approximations. I think GAMS I believe both AMPL and GAMS have trial versions, and they are free for students with small problems (<300 variables). Another option is AIMMS, but they only support Windows and Linux (I have a Mac).

As for references, the following books have helped me in the past:

  • "Numerical Optimization" by Nocedal and Wright.
  • "Practical Optimization" by Gill, Murray and Wright.
  • "Practical Methods of Optimization" by Fletcher.
  • "Iterative Methods for Linear and Nonlinear Equations" by Kelley available for free on the site of SIAM, the publisher. There is also a similar one about optimization.
  • "Solving Nonlinear Equations with Newton's Method", also by Kelley, with Matlab code.

Good luck. And next time, let us know if the problem is smooth, polynomial, etc. Not everyone will be familiar with the field/application where the problem/equations are coming from (like me, coming from an Economics/Math background), but they may still help you anyway.

Edit: the following is a very good reference on Nonlinear Programming methods and software that might be useful. I know you are solving system of equations, but you can always an optimization solver to solve a feasibility problem (constant objective, thus solving only the constraints given by your nonlinear system) or, as you have degrees of freedom, impose some objective (like norm minimization).

Two other useful websites (again, in optimization, but again, it may be useful, for reasons already stated):

  • Benchmarks of different solvers by Hans Mittemman
  • The NEOS Optimization Guide. An overview of algorithms, solvers, etc.
  • The NEOS Server. A place where you can freely submit your optimization problems to be solved by various different solvers. Supports problems modeled mostly in AMPL or GAMS, but a few solvers also support other languages.

The NEOS server should allow you to experiment with different algorithms/software to solve your problem. The first three items above should give you some guidance in which methods to choose. Hope it helps.

share|cite|improve this answer
Thanks for the input! I am dealing with non-smooth non-linearities. But mostly I try approximating these by some smooth functions for now. Trying to go for a true non-smooth system seems to cause a lot of issues. – Markus Oct 30 '12 at 14:12

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.