Source: Ann. Appl. Probab.
Volume 9, Number 1
We consider stochastic approximation algorithms with constant step
size whose average ordinary differential equation (ODE) is cooperative and
irreducible. We show that, under mild conditions on the noise process,
invariant measures and empirical occupations measures of the process weakly
converge (as the time goes to infinity and the step size goes to zero) toward
measures which are supported by stable equilibria of the ODE. These results are
applied to analyzing the long-term behavior of a class of learning processes
arising in game theory.
Azencott, R. and Ruget, G. (1977). M´elanges d'´equations differentielles et grands ´ecarts a la loi des grands nombres. Z. Wahrsch. Verw. Gebiete 38 1-54. Bena¨im, M. (1996a). A dy namical sy stem approach to stochastic approximations. Siam J. Control Optim. 34 437-472.
Bena¨im, M. (1998). Recursive algorithms, urn processes and chaining number of chain recurrent sets. Ergodic Theory Dy nam. Sy stems 18 53-87.
Bena¨im, M. and Hirsch, M. W. (1995). Dy namics of Morse Smale urn processes. Ergodic Theory Dy nam. Sy stems 15 1005-1030.
Bena¨im, M. and Hirsch, M. W. (1996). Asy mptotic pseudo-trajectories and chain-recurrent flows, with applications. J. Dy nam. Differential Equations 8 141-174.
Bena¨im, M. and Hirsch, M. W. (1997). Information games, Dy namical sy stems and stochastic approximation. Discussion paper for IIASA workshop Oct 17-20.
Benveniste, A. M´etivier, M. and Priouret, P. (1990). Stochastic Approximations and Adaptive Algorithms. Springer, New York.
Chow, S. N. and Hale, J. (1982). Methods of Bifurcation Theory. Springer, New York.
Mathematical Reviews (MathSciNet): MR660633
Conley, C. C. (1978). Isolated Invariant Sets and the Morse Index. Amer. Math. Soc., Providence, RI.
Duflo, M. (1996). Algorithmes Stochastiques. Springer, Berlin.
Duflo, M. (1997). Random Iterative Models. Springer, New York.
Dupuis, P. (1988). Large deviations analysis of some recursive algorithms with state dependent noise. Ann. Probab. 6 1509-1536.
Dupuis, P. and Ellis R. S. (1997). A Weak Convergence Approach to the Theory of Large Deviations. Wiley, New York.
Fort, J. C. and Pages, G. (1996). Asy mptotics behaviour of a Markovian constant step size algorithm. Siam J. Control Optim. To appear.
Freidlin, M. I. (1978). The averaging principle and theorems on large deviations. Russian Math. Survey s 33 117-176.
Freidlin, M. I. and Wentzell, A. D. (1984). Random Perturbation of Dy namical Sy stems. Springer, New York.
Fudenberg, D. and Levine, D. (1998). Theory of Learning in Games. MIT Press.
Hirsch, M. W. (1985). Sy stems of differential equations that are competitive or cooperative II: convergence almost every where. SIAM J. Math. Anal. 16 423-439. Hirsch, M. W. (1988a). Sy stems of differential equations which are competitive or cooperative III: competing species. Nonlinearity 1 51-71. Hirsch, M. W. (1988b). Stability and convergence in strongly monotone dy namical sy stems. J. Reine Angew. Math. 383 1-53.
Hirsch, M. W. (1996). Chain transitive sets for smooth strongly monotone maps. Preprint, Univ. California, Berkeley.
Jiang, J.-F. (1991). Attractors for strongly monotone flows. J. Math. Anal. Appl. 162 210-222.
Jiang, J.-F. and Yu, S.-X. (1995). Stable cy cles for attractors of strongly monotone discrete-time dy namical sy stems. Comm. Appl. Nonlinear Anal. 2 455-458.
Kifer, Y. (1988). Random Perturbation of Dy namical Sy stems. Birkh¨auser, Boston.
Kunze, H. and Siegel, D. (1994). A graph theoretic approach to monotonicity with respect to initial conditions. In Comparison Methods and Stability Theory (X. Liu and D. Siegel, eds.) Dekker, New York.
Kushner, H. J. and Clark, D. S. (1978). Stochastic Approximation Methods for Constrained and Unconstrained Sy stems. Springer, New York.
Ljung, L. (1977). Analy sis of recursive stochastic algorithms. IEEE Trans. Automat. Control AC22 551-575.
Mierczy ´nski, J. (1994). P-arcs in strongly monotone discrete-time dy namical sy stems. Differential Integral Equations 7 1473-1494.
Smith, H. (1995). Monotone Dy namical Sy stems. Amer. Math. Soc., Providence, RI.
Tak´a c, P. (1992). Domains of attraction of generic -limit sets for strongly monotone discretetime semigroups. J. Reine Angew. Math. 423 101-173.
Tere s c´ak, I. (1994). Dy namics of C1 smooth strongly monotone discrete-time dy namical sy stems. Preprint.
Varadhan, (1984). Large Deviations and Applications. SIAM, Philadelphia.
Weibull, J. W. (1995). Evolutionary Game Theory. MIT Press.