## Annals of Applied Probability

### Singularly perturbed Markov chains: Limit results and applications

#### Abstract

This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale scenarios naturally arise. One of the important questions is: As the rate of fluctuation of the Markov chain goes to infinity, if the limit distributions of suitably centered and scaled sequences of occupation measures exist, what can be said about the convergence rate? By combining singular perturbation techniques and probabilistic methods, this paper addresses the issue by concentrating on sequences of centered and scaled functional occupation processes. The results obtained are then applied to treat a queueing system example.

#### Article information

Source
Ann. Appl. Probab., Volume 17, Number 1 (2007), 207-229.

Dates
First available in Project Euclid: 13 February 2007

https://projecteuclid.org/euclid.aoap/1171377182

Digital Object Identifier
doi:10.1214/105051606000000682

Mathematical Reviews number (MathSciNet)
MR2292585

Zentralblatt MATH identifier
1138.60331

#### Citation

Yin, George; Zhang, Hanqin. Singularly perturbed Markov chains: Limit results and applications. Ann. Appl. Probab. 17 (2007), no. 1, 207--229. doi:10.1214/105051606000000682. https://projecteuclid.org/euclid.aoap/1171377182

#### References

• Altman, E., Avrachenkov, K. E. and Nunez-Queija, R. (2004). Perturbation analysis for denumerable Markov chains with applications to queueing models. Adv. in Appl. Probab. 36 839–853.
• Bogoliubov, N. N. and Mitropolskii, Y. A. (1961). Asymptotic Methods in the Theory of Nonlinear Oscillator. Gordon and Breach, New York.
• Chow, Y. S. and Teicher, H. (1988). Probability Theory, 2nd ed. Springer, New York.
• Csáki, E., Csörgö, M., Földes, A. and Révész, P. (1992). Strong approximation of additive functionals. J. Theoret. Probab. 5 679–706.
• Csörgö, M., Deheuvels, P. and Horváth, L. (1987). An approximation of stopped sums with applications in queueing theory. Adv. in Appl. Probab. 19 674–690.
• Csörgö, M., Horváth, L. and Steinebach, J. (1987). Invariance principles for renewal processes. Ann. Probab. 15 1141–1160.
• Csörgö, M. and Révész, P. (1975). A new method to prove Strassen type laws of invariance principles, I and II. Z. Wahrsch. Verw. Gebiete 31 255–269.
• Csörgö, M. and Révész, P. (1981). Strong Approximations in Probability and Statistics. Academic Press, New York.
• Davis, M. H. A. (1993). Markov Models and Optimization. Chapman and Hall, London.
• de Souza e Silva, E. and Gail, H. R. (1986). Calculating cumulative operational time distributions of repairable computer systems. IEEE Trans. Computers 35 322–332.
• Doob, J. L. (1990). Stochastic Processes. Wiley, New York.
• Fouque, J. P., Papanicolaou, G. and Sircar, R. K. (2000). Derivatives in Financial Markets with Stochastic Volatility. Cambridge Univ. Press.
• Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Application. Academic Press, New York.
• Hutson, V. and Pym, J. S. (1980). Applications of Functional Analysis and Operator Theory. Academic Press, London.
• Ikeda, N. and Watanabe, S. (1981). Stochastic Differential Equations and Diffusion Processes. North-Holland, Amsterdam.
• Keller, J. B. (1982). Time-dependent queues. SIAM Rev. 24 401–412.
• Kiefer, J. (1972). Skorohod embedding of multivariate RV's and the sample DF. Z. Wahrsch. Verw. Gebiete 24 1–35.
• Khasminskii, R. Z., Yin, G. and Zhang, Q. (1996). Asymptotic expansions of singularly perturbed systems involving rapidly fluctuating Markov chains. SIAM J. Appl. Math. 56 277–293.
• Komlós, J., Major, P. and Tusnády, G. (1975). An approximation of partial sums of independent R.V.'s and the sample DF. I. Z. Wahrsch. Verw. Gebiete 32 111–131.
• Krichagina, V., Luo, S., Sethi, S. and Taksar, M. (1993). Production control in a failure-prone manufacturing system: Diffusion approximation and asymptotic optimality. Ann. Appl. Probab. 3 421–453.
• Kuelbs, J. (1973). The invariance principle for Banach space valued random variables. J. Multivariate Anal. 3 161–172.
• Kushner, H. J. (1990). Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems. Birkhäuser, Boston.
• Kurtz, T. G. (1978). Strong approximation theorems for density dependent Markov chains. Stochastic Process. Appl. 6 223–240.
• Massey, W. (1985). Asymptotic analysis of the time dependent $M/M/1$ queue. Math. Oper. Res. 10 305–327.
• Massey, W. A. and Whitt, W. (1998). Uniform acceleration expansions for Markov chains with time-varying rates. Ann. Appl. Probab. 8 1130–1155.
• Rosenkrantz, W. (1980). On the accuracy of Kingman's heavy traffic approximation in the theory of queues. Z. Wahrsch. Verw. Gebiete 51 115–121.
• Stout, W. F. (1974). Almost Sure Convergence. Academic Press, New York.
• Sethi, S. P. and Zhang, Q. (1994). Hierarchical Decision Making in Stochastic Manufacturing Systems. Birkhäuser, Boston.
• Strassen, V. (1967). Almost sure behavior of sums of independent random variables and martingales. Proc. 5th Berkeley Symp. Math. Statist. Probab. II 315–343. Univ. California Press, Berkeley.
• Yin, G. and Zhang, H. (2005). Two-time-scale Markov chains and applications to quasi-birth–death queues. SIAM J. Appl. Math. 65 567–586.
• Yin, G. and Zhang, Q. (1998). Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach. Springer, New York.
• Yin, G., Zhang, Q. and Badowski, G. (2000). Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states. Ann. Appl. Probab. 10 549–572.