Zangwill's well-known global convergence theorem (Zangwill, W. I. 1969. Nonlinear Programming: a Unified Approach. Englewood-Cliffs, N.J.: Prentice-Hall) provides sufficient conditions under which global convergence can be proven for generic deterministic descent algorithms.

Is there an analogous result that applies to stochastic optimisation algorithms? In particular, to online stochastic gradient descent, and mini-batch stochastic gradient descent?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.