Annals of Applied Probability

Singularly perturbed Markov chains: Limit results and applications

George Yin and Hanqin Zhang

Full-text: Open access


This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale scenarios naturally arise. One of the important questions is: As the rate of fluctuation of the Markov chain goes to infinity, if the limit distributions of suitably centered and scaled sequences of occupation measures exist, what can be said about the convergence rate? By combining singular perturbation techniques and probabilistic methods, this paper addresses the issue by concentrating on sequences of centered and scaled functional occupation processes. The results obtained are then applied to treat a queueing system example.

Article information

Ann. Appl. Probab., Volume 17, Number 1 (2007), 207-229.

First available in Project Euclid: 13 February 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 34E05: Asymptotic expansions 60F17: Functional limit theorems; invariance principles 60J27: Continuous-time Markov processes on discrete state spaces

Singular perturbation Markov chain asymptotic expansion occupation measure diffusion process


Yin, George; Zhang, Hanqin. Singularly perturbed Markov chains: Limit results and applications. Ann. Appl. Probab. 17 (2007), no. 1, 207--229. doi:10.1214/105051606000000682.

Export citation


  • Altman, E., Avrachenkov, K. E. and Nunez-Queija, R. (2004). Perturbation analysis for denumerable Markov chains with applications to queueing models. Adv. in Appl. Probab. 36 839–853.
  • Bogoliubov, N. N. and Mitropolskii, Y. A. (1961). Asymptotic Methods in the Theory of Nonlinear Oscillator. Gordon and Breach, New York.
  • Chow, Y. S. and Teicher, H. (1988). Probability Theory, 2nd ed. Springer, New York.
  • Csáki, E., Csörgö, M., Földes, A. and Révész, P. (1992). Strong approximation of additive functionals. J. Theoret. Probab. 5 679–706.
  • Csörgö, M., Deheuvels, P. and Horváth, L. (1987). An approximation of stopped sums with applications in queueing theory. Adv. in Appl. Probab. 19 674–690.
  • Csörgö, M., Horváth, L. and Steinebach, J. (1987). Invariance principles for renewal processes. Ann. Probab. 15 1141–1160.
  • Csörgö, M. and Révész, P. (1975). A new method to prove Strassen type laws of invariance principles, I and II. Z. Wahrsch. Verw. Gebiete 31 255–269.
  • Csörgö, M. and Révész, P. (1981). Strong Approximations in Probability and Statistics. Academic Press, New York.
  • Davis, M. H. A. (1993). Markov Models and Optimization. Chapman and Hall, London.
  • de Souza e Silva, E. and Gail, H. R. (1986). Calculating cumulative operational time distributions of repairable computer systems. IEEE Trans. Computers 35 322–332.
  • Doob, J. L. (1990). Stochastic Processes. Wiley, New York.
  • Fouque, J. P., Papanicolaou, G. and Sircar, R. K. (2000). Derivatives in Financial Markets with Stochastic Volatility. Cambridge Univ. Press.
  • Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Application. Academic Press, New York.
  • Hutson, V. and Pym, J. S. (1980). Applications of Functional Analysis and Operator Theory. Academic Press, London.
  • Ikeda, N. and Watanabe, S. (1981). Stochastic Differential Equations and Diffusion Processes. North-Holland, Amsterdam.
  • Keller, J. B. (1982). Time-dependent queues. SIAM Rev. 24 401–412.
  • Kiefer, J. (1972). Skorohod embedding of multivariate RV's and the sample DF. Z. Wahrsch. Verw. Gebiete 24 1–35.
  • Khasminskii, R. Z., Yin, G. and Zhang, Q. (1996). Asymptotic expansions of singularly perturbed systems involving rapidly fluctuating Markov chains. SIAM J. Appl. Math. 56 277–293.
  • Komlós, J., Major, P. and Tusnády, G. (1975). An approximation of partial sums of independent R.V.'s and the sample DF. I. Z. Wahrsch. Verw. Gebiete 32 111–131.
  • Krichagina, V., Luo, S., Sethi, S. and Taksar, M. (1993). Production control in a failure-prone manufacturing system: Diffusion approximation and asymptotic optimality. Ann. Appl. Probab. 3 421–453.
  • Kuelbs, J. (1973). The invariance principle for Banach space valued random variables. J. Multivariate Anal. 3 161–172.
  • Kushner, H. J. (1990). Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems. Birkhäuser, Boston.
  • Kurtz, T. G. (1978). Strong approximation theorems for density dependent Markov chains. Stochastic Process. Appl. 6 223–240.
  • Massey, W. (1985). Asymptotic analysis of the time dependent $M/M/1$ queue. Math. Oper. Res. 10 305–327.
  • Massey, W. A. and Whitt, W. (1998). Uniform acceleration expansions for Markov chains with time-varying rates. Ann. Appl. Probab. 8 1130–1155.
  • Rosenkrantz, W. (1980). On the accuracy of Kingman's heavy traffic approximation in the theory of queues. Z. Wahrsch. Verw. Gebiete 51 115–121.
  • Stout, W. F. (1974). Almost Sure Convergence. Academic Press, New York.
  • Sethi, S. P. and Zhang, Q. (1994). Hierarchical Decision Making in Stochastic Manufacturing Systems. Birkhäuser, Boston.
  • Strassen, V. (1967). Almost sure behavior of sums of independent random variables and martingales. Proc. 5th Berkeley Symp. Math. Statist. Probab. II 315–343. Univ. California Press, Berkeley.
  • Yin, G. and Zhang, H. (2005). Two-time-scale Markov chains and applications to quasi-birth–death queues. SIAM J. Appl. Math. 65 567–586.
  • Yin, G. and Zhang, Q. (1998). Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach. Springer, New York.
  • Yin, G., Zhang, Q. and Badowski, G. (2000). Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states. Ann. Appl. Probab. 10 549–572.