Probability Surveys

Differential equation approximations for Markov chains

R.W.R. Darling and J.R. Norris

Full-text: Open access

Abstract

We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is emphasised. The general theory is illustrated in three examples: the classical stochastic epidemic, a population process model with fast and slow variables, and core-finding algorithms for large random hypergraphs.

Article information

Source
Probab. Surveys, Volume 5 (2008), 37-79.

Dates
First available in Project Euclid: 23 April 2008

Permanent link to this document
https://projecteuclid.org/euclid.ps/1208958281

Digital Object Identifier
doi:10.1214/07-PS121

Mathematical Reviews number (MathSciNet)
MR2395153

Zentralblatt MATH identifier
1189.60152

Subjects
Primary: 05C65: Hypergraphs
Secondary: 60J75: Jump processes 05C80: Random graphs [See also 60B20]

Citation

Darling, R.W.R.; Norris, J.R. Differential equation approximations for Markov chains. Probab. Surveys 5 (2008), 37--79. doi:10.1214/07-PS121. https://projecteuclid.org/euclid.ps/1208958281


Export citation

References

  • [1] Dimitris Achlioptas. Lower bounds for random 3-SAT via differential equations. Theoret. Comput. Sci., 265(1-2):159–185, 2001. Phase transitions in combinatorial problems (Trieste, 1999).
  • [2] Karen Ball, Thomas G. Kurtz, Lea Popovic, and Greg Rempala. Asymptotic analysis of multiscale approximations to reaction networks. Ann. Appl. Probab., 16(4):1925–1961, 2006.
  • [3] D. J. Daley and J. Gani. Epidemic modelling: an introduction, volume 15 of Cambridge Studies in Mathematical Biology. Cambridge University Press, Cambridge, 1999.
  • [4] R. W. R. Darling and J. R. Norris. Structure of large random hypergraphs. Ann. Appl. Probab., 15(1A):125–152, 2005.
  • [5] R. W. R. Darling and J. R. Norris. Cores and cycles in random hypergraphs. In preparation, 2008.
  • [6] Stewart N. Ethier and Thomas G. Kurtz. Markov processes. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley & Sons Inc., New York, 1986.
  • [7] Bruce Hajek. Asymptotic analysis of an assignment problem arising in a distributed communications protocol. In Proc. of the 27th Conf. on Decision and Control, pages 1455–1459. IEEE Press, 1988.
  • [8] Jean Jacod and Albert N. Shiryaev. Limit theorems for stochastic processes, volume 288 of Grundlehren der Mathematischen Wissenschaften. Springer-Verlag, Berlin, 2003.
  • [9] Olav Kallenberg. Foundations of modern probability. Probability and its Applications (New York). Springer-Verlag, New York, second edition, 2002.
  • [10] Richard Karp and Michael Sipser. Maximum matchings in sparse random graphs. In 22nd Annual Symposium on Foundations of Computer Science, pages 364–375. IEEE Press, 1981.
  • [11] Claude Kipnis and Claudio Landim. Scaling limits of interacting particle systems, volume 320 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Springer-Verlag, Berlin, 1999.
  • [12] Thomas G. Kurtz. Approximation of population processes, volume 36 of CBMS-NSF Regional Conference Series in Applied Mathematics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, Pa., 1981.
  • [13] Harold J. Kushner and G. George Yin. Stochastic approximation and recursive algorithms and applications. Springer, New York, 2003.
  • [14] Malwina J. Luczak and James Norris. Strong approximation for the supermarket model. Ann. Appl. Probab., 15(3):2038–2061, 2005.
  • [15] Brendan D. McKay. Asymptotics for 0-1 matrices with prescribed line sums. In Enumeration and design (Waterloo, Ont., 1982), pages 225–238. Academic Press, Toronto, ON, 1984.
  • [16] Michael Mitzenmacher. Studying balanced allocations with differential equations. Combin. Probab. Comput., 8(5):473–482, 1999.
  • [17] Michael Molloy. Cores in random hypergraphs and Boolean formulas. Random Structures Algorithms, 27(1):124–135, 2005.
  • [18] Rémi Monasson and Guilhelm Semerjian. Relaxation and metastability in a local search procedure for the random satisfiability problem. Physical Review E, 67, 2003.
  • [19] Boris Pittel, Joel Spencer, and Nicholas Wormald. Sudden emergence of a giant k-core in a random graph. J. Combin. Theory Ser. B, 67(1):111–151, 1996.
  • [20] Oliver Riordan. The k-core and branching processes. Combin. Probab. Comput., 17:111–136, 2008.
  • [21] Adam Shwartz and Alan Weiss. Large deviations for performance analysis. Stochastic Modeling Series. Chapman & Hall, London, 1995. Queues, communications, and computing, With an appendix by Robert J. Vanderbei.
  • [22] H. F. Trotter. Approximation of semi-groups of operators. Pacific Journal of Mathematics, 8:887–919, 1958.
  • [23] Amanda G. Turner. Convergence of Markov processes near saddle fixed points. Ann. Probab., 35(3):1141–1171, 2007.
  • [24] Nicholas C. Wormald. Differential equations for random processes and random graphs. Ann. Appl. Probab., 5(4):1217–1235, 1995.