Convergence with probability one of a recursive stochastic approximation algorithm is considered. Some extensions of previous results for the Robbins-Monro and the Kiefer-Wolfowitz procedures are given. An inportant feature of the approach taken here is that the convergence analysis can be directly extended to more complex algorithms.
"Strong Convergence of a Stochastic Approximation Algorithm." Ann. Statist. 6 (3) 680 - 696, May, 1978. https://doi.org/10.1214/aos/1176344212