I have been reading a book on Reinforcement Learning and the author mentions "a well-known result in stochastic approximation theory" that gives us the conditions required to assure convergence in probability 1. Screenshot of these conditions below:

enter image description here

What is this well-known result? I've been trying to google around for it but am having trouble finding it.

$\ $

Edit for additional clarity

Here the $\alpha_n$ are step size parameters for processing rewards. You can see them being defined on p.25 (as it appears on the book page not page count in a PDF reader) of the book, with the below screenshot taken from p.26.

  • $\begingroup$ How can we locate the appropriate portions of the huge book to find the definitions of $\alpha_n(a)$? By no means are these standard notations. $\endgroup$ Sep 8, 2018 at 23:29

1 Answer 1


In a lecture from David Silver, he mentions that these are the conditions for the Robbins-Monro algorithm to converge, which is what I was looking for.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .