How to define the characteristic length scale in a downhill simplex method? I am currently converting a minimization problem from Matlab to C++, using the Numerical Recipes implementation of the Nelder and Mead Downhill simplex method. The function requires me to define a constant lambda for each variable, which represents the "characteristic length scale" of the variable.
I tried to find a more formal definition of what the authors mean by this, and how to choose one, but couldn't find anything. I'm guessing it's something of a "choose something that looks like a reasonable step size for that variable and hope that it converges fast enough; otherwise, try something else". Any pointers to something a little bit more scientific?
 A: For the purposes of the Nelder-Mead algorithm, the characteristic length scale for a particular variable is basically your best guess as to the size of the potential solution space in that variable.
(Example taken from this site.)  For example, in a 3-dimensional problem, if the initial guess is the point $[0,0,0]$, and you know that the function's minimum value occurs in the interval
$-10 < x_0 < 10,$
$-100 < x_1 < 100,$
$-200 < x_2 < 200,$
then you could set $\lambda_0$, $\lambda_1$, and $\lambda_2$ to 10, 100, and 200, respectively. 
I suppose this quantity can be called a step size, but it's only used in the initialization part of the algorithm.   It's not (which is how I normally think of step sizes) used in each iteration of the algorithm to step from one solution to the next.  Those step sizes are the reflection, expansion, contraction, and shrinking parameters.
Some additional references:


*

*Nelder and Mead's original paper.

*This paper by Lagarias, et al, contains a nice presentation of the Nelder-Mead algorithm.

*A Nelder Mead User's Manual.  In particular, see the discussion of different ways to form the initial simplex.
