Stochastic approximation, cooperative dynamics and supermodular games

Abstract

This paper considers a stochastic approximation algorithm, with decreasing step size and martingale difference noise. Under very mild assumptions, we prove the nonconvergence of this process toward a certain class of repulsive sets for the associated ordinary differential equation (ODE). We then use this result to derive the convergence of the process when the ODE is cooperative in the sense of Hirsch [SIAM J. Math. Anal. 16 (1985) 423–439]. In particular, this allows us to extend significantly the main result of Hofbauer and Sandholm [Econometrica 70 (2002) 2265–2294] on the convergence of stochastic fictitious play in supermodular games.