# Definition of Global Convergence

I am confused by the notion of "global convergence" as used in numerical optimization literature, and did not find an exact definition for that yet. Now I try to double check my understanding here.

It clearly is NOT concerned with convergence to the global minimum. Does it mean convergence to a local minimum regardless of the initial point? (How about local maximum, saddle point?) What methods do not have global convergence? Those that with selection of bad initial point can stop in an arbitrary non-critical point?

Also, a globally convergence method applied on a convex function, gives the global minima, right?

See the definition in http://papers.nips.cc/paper/3646-on-the-convergence-of-the-concave-convex-procedure.pdf. The points generated by an algorithm with this property

• converge for any initial point.
• converge to a stationary point.

Global convergence is generally used in the context of Iterative numerical algorithms. It is usually defined as a sequence generated by the iterative algorithm converges to a solution point. This is important as it answers the question whether a particular algorithm, when initiated at a point far from the solution point, converges to it.

• What is a considered a solution for a numerical optimization problem? Aug 21, 2014 at 19:16

There is an indirect definition in the book Numeric Optimization. In page 40, it is saying

We use the term globally convergent to refer to algorithms for which the property (3.18) is satisfied, but note that this term is sometimes used in other contexts to mean different things.

where (3.18) is exactly $$\lim _{k \rightarrow \infty}\left\|\nabla f_{k}\right\|=0$$ given the sequence $$\{x_k\}$$. So in this context, globally convergent is exactly as The Pheromone Kid's answer. But there might be a different meaning in different context as the book suggests.

As far as I know optimisation would be the search for maxima/minima.

If you'd imagine your function being a mountain chain, the optimum would equal a peak or a valley.

To find the minima or maxima you could calculate the gradient of the function and starting from an initial point change the values of this initial point into the direction the gradient is pointing in until you reach the peak/valley, which would -- in most cases -- be a local optimum.

Global convergence would hence be convergence to the global optimum regardless of the initial starting point.

Hope this helps.

Edit: Global Convergence is indeed the convergence to Minima/Maxima regardless of the initial starting point.

• I disagree. I believe what is regarded as solution is the Wikipedia page is a local solution. So global convergence does not worry about convergence to global optimum in the numerical optimization literature, but to a local optimum. Aug 21, 2014 at 19:12
• @user25004 Thx for pointing this out Sep 2, 2014 at 17:00