Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am currently converting a minimization problem from Matlab to C++, using the Numerical Recipes implementation of the Nelder and Mead Downhill simplex method. The function requires me to define a constant lambda for each variable, which represents the "characteristic length scale" of the variable.

I tried to find a more formal definition of what the authors mean by this, and how to choose one, but couldn't find anything. I'm guessing it's something of a "choose something that looks like a reasonable step size for that variable and hope that it converges fast enough; otherwise, try something else". Any pointers to something a little bit more scientific?

share|cite|improve this question
up vote 4 down vote accepted

For the purposes of the Nelder-Mead algorithm, the characteristic length scale for a particular variable is basically your best guess as to the size of the potential solution space in that variable.

(Example taken from this site.) For example, in a 3-dimensional problem, if the initial guess is the point $[0,0,0]$, and you know that the function's minimum value occurs in the interval

$-10 < x_0 < 10,$

$-100 < x_1 < 100,$

$-200 < x_2 < 200,$

then you could set $\lambda_0$, $\lambda_1$, and $\lambda_2$ to 10, 100, and 200, respectively.

I suppose this quantity can be called a step size, but it's only used in the initialization part of the algorithm. It's not (which is how I normally think of step sizes) used in each iteration of the algorithm to step from one solution to the next. Those step sizes are the reflection, expansion, contraction, and shrinking parameters.

Some additional references:

  1. Nelder and Mead's original paper.

  2. This paper by Lagarias, et al, contains a nice presentation of the Nelder-Mead algorithm.

  3. A Nelder Mead User's Manual. In particular, see the discussion of different ways to form the initial simplex.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.