Differential equation approximations for Markov chains
R.W.R. Darling and J.R. Norris
Source: Probab. Surveys Volume 5 (2008), 37-79.
Abstract
We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is emphasised. The general theory is illustrated in three examples: the classical stochastic epidemic, a population process model with fast and slow variables, and core-finding algorithms for large random hypergraphs.
Primary Subjects: 05C65
Secondary Subjects: 60J75, 05C80
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.ps/1208958281
Digital Object Identifier: doi:10.1214/07-PS121
Mathematical Reviews number (MathSciNet):
MR2395153
References
[1] Dimitris Achlioptas. Lower bounds for random 3-SAT via differential equations. Theoret. Comput. Sci., 265(1-2):159–185, 2001. Phase transitions in combinatorial problems (Trieste, 1999).
Mathematical Reviews (MathSciNet):
MR1848217
Digital Object Identifier: doi:10.1016/S0304-3975(01)00159-1
[2] Karen Ball, Thomas G. Kurtz, Lea Popovic, and Greg Rempala. Asymptotic analysis of multiscale approximations to reaction networks. Ann. Appl. Probab., 16(4):1925–1961, 2006.
Mathematical Reviews (MathSciNet):
MR2288709
Digital Object Identifier: doi:10.1214/105051606000000420
Project Euclid: euclid.aoap/1169065212
[3] D. J. Daley and J. Gani. Epidemic modelling: an introduction, volume 15 of Cambridge Studies in Mathematical Biology. Cambridge University Press, Cambridge, 1999.
Mathematical Reviews (MathSciNet):
MR1688203
[4] R. W. R. Darling and J. R. Norris. Structure of large random hypergraphs. Ann. Appl. Probab., 15(1A):125–152, 2005.
Mathematical Reviews (MathSciNet):
MR2115039
Digital Object Identifier: doi:10.1214/105051604000000567
Project Euclid: euclid.aoap/1106922324
[5] R. W. R. Darling and J. R. Norris. Cores and cycles in random hypergraphs. In preparation, 2008.
Mathematical Reviews (MathSciNet):
MR2395153
Digital Object Identifier: doi:10.1214/07-PS121
Project Euclid: euclid.ps/1208958281
[6] Stewart N. Ethier and Thomas G. Kurtz. Markov processes. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley & Sons Inc., New York, 1986.
Mathematical Reviews (MathSciNet):
MR838085
[7] Bruce Hajek. Asymptotic analysis of an assignment problem arising in a distributed communications protocol. In Proc. of the 27th Conf. on Decision and Control, pages 1455–1459. IEEE Press, 1988.
[8] Jean Jacod and Albert N. Shiryaev. Limit theorems for stochastic processes, volume 288 of Grundlehren der Mathematischen Wissenschaften. Springer-Verlag, Berlin, 2003.
Mathematical Reviews (MathSciNet):
MR1943877
[9] Olav Kallenberg. Foundations of modern probability. Probability and its Applications (New York). Springer-Verlag, New York, second edition, 2002.
Mathematical Reviews (MathSciNet):
MR1876169
[10] Richard Karp and Michael Sipser. Maximum matchings in sparse random graphs. In 22nd Annual Symposium on Foundations of Computer Science, pages 364–375. IEEE Press, 1981.
[11] Claude Kipnis and Claudio Landim. Scaling limits of interacting particle systems, volume 320 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Springer-Verlag, Berlin, 1999.
Mathematical Reviews (MathSciNet):
MR1707314
[12] Thomas G. Kurtz. Approximation of population processes, volume 36 of CBMS-NSF Regional Conference Series in Applied Mathematics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, Pa., 1981.
Mathematical Reviews (MathSciNet):
MR610982
[13] Harold J. Kushner and G. George Yin. Stochastic approximation and recursive algorithms and applications. Springer, New York, 2003.
Mathematical Reviews (MathSciNet):
MR1993642
[14] Malwina J. Luczak and James Norris. Strong approximation for the supermarket model. Ann. Appl. Probab., 15(3):2038–2061, 2005.
Mathematical Reviews (MathSciNet):
MR2152252
Digital Object Identifier: doi:10.1214/105051605000000368
Project Euclid: euclid.aoap/1121433776
[15] Brendan D. McKay. Asymptotics for 0-1 matrices with prescribed line sums. In Enumeration and design (Waterloo, Ont., 1982), pages 225–238. Academic Press, Toronto, ON, 1984.
Mathematical Reviews (MathSciNet):
MR782316
[16] Michael Mitzenmacher. Studying balanced allocations with differential equations. Combin. Probab. Comput., 8(5):473–482, 1999.
Mathematical Reviews (MathSciNet):
MR1731982
Digital Object Identifier: doi:10.1017/S0963548399003946
[17] Michael Molloy. Cores in random hypergraphs and Boolean formulas. Random Structures Algorithms, 27(1):124–135, 2005.
Mathematical Reviews (MathSciNet):
MR2150018
[18] Rémi Monasson and Guilhelm Semerjian. Relaxation and metastability in a local search procedure for the random satisfiability problem. Physical Review E, 67, 2003.
[19] Boris Pittel, Joel Spencer, and Nicholas Wormald. Sudden emergence of a giant k-core in a random graph. J. Combin. Theory Ser. B, 67(1):111–151, 1996.
Mathematical Reviews (MathSciNet):
MR1385386
Digital Object Identifier: doi:10.1006/jctb.1996.0036
[20] Oliver Riordan. The k-core and branching processes. Combin. Probab. Comput., 17:111–136, 2008.
Mathematical Reviews (MathSciNet):
MR2376426
Digital Object Identifier: doi:10.1017/S0963548307008589
[21] Adam Shwartz and Alan Weiss. Large deviations for performance analysis. Stochastic Modeling Series. Chapman & Hall, London, 1995. Queues, communications, and computing, With an appendix by Robert J. Vanderbei.
Mathematical Reviews (MathSciNet):
MR1335456
[22] H. F. Trotter. Approximation of semi-groups of operators. Pacific Journal of Mathematics, 8:887–919, 1958.
Mathematical Reviews (MathSciNet):
MR103420
Project Euclid: euclid.pjm/1103039709
[23] Amanda G. Turner. Convergence of Markov processes near saddle fixed points. Ann. Probab., 35(3):1141–1171, 2007.
Mathematical Reviews (MathSciNet):
MR2319718
Digital Object Identifier: doi:10.1214/009117906000000836
Project Euclid: euclid.aop/1178804325
[24] Nicholas C. Wormald. Differential equations for random processes and random graphs. Ann. Appl. Probab., 5(4):1217–1235, 1995.
Mathematical Reviews (MathSciNet):
MR1384372
Digital Object Identifier: doi:10.1214/aoap/1177004612
Project Euclid: euclid.aoap/1177004612
