The Annals of Applied Probability

Uniform Markov renewal theory and ruin probabilities in Markov random walks

Cheng-Der Fuh

Full-text: Open access


Let {Xn,n0} be a Markov chain on a general state space ${\mathcal{X}}$ with transition probability P and stationary probability π. Suppose an additive component Sn takes values in the real line R and is adjoined to the chain such that {(Xn,Sn),n0} is a Markov random walk. In this paper, we prove a uniform Markov renewal theorem with an estimate on the rate of convergence. This result is applied to boundary crossing problems for {(Xn,Sn),n0}. To be more precise, for given b0, define the stopping time τ=τ(b)=inf {n:Sn>b}. When a drift μ of the random walk Sn is 0, we derive a one-term Edgeworth type asymptotic expansion for the first passage probabilities Pπ{τ<m} and Pπ{τ<m,Sm<c}, where m, cb and Pπ denotes the probability under the initial distribution π. When μ0, Brownian approximations for the first passage probabilities with correction terms are derived. Applications to sequential estimation and truncated tests in random coefficient models and first passage times in products of random matrices are also given.

Article information

Ann. Appl. Probab., Volume 14, Number 3 (2004), 1202-1241.

First available in Project Euclid: 13 July 2004

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K05: Renewal theory
Secondary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces) 60K15: Markov renewal processes, semi-Markov processes

Brownian approximation first passage probabilities ladder height distribution Markov-dependent Wald martingale products of random matrices random coefficient models uniform Markov renewal theory


Fuh, Cheng-Der. Uniform Markov renewal theory and ruin probabilities in Markov random walks. Ann. Appl. Probab. 14 (2004), no. 3, 1202--1241. doi:10.1214/105051604000000260.

Export citation


  • Alsmeyer, G. (1994). On the Markov renewal theorem. Stochastic Process. Appl. 50 37–56.
  • Alsmeyer, G. (2000). The ladder variables of a Markov random walk. Theory Probab. Math. Statist. 20 151–168.
  • Arndt, K. (1980). Asymptotic properties of the distribution of the supremum of a random walk on a Markov chain. Theory Probab. Appl. 25 309–323.
  • Asmussen, S. (1989a). Aspects of matrix Wiener–Hopf factorization in applied probability. Math. Sci. 14 101–116.
  • Asmussen, S. (1989b). Risk theory in a Markovian environment. Scand. Actuar. J. 69–100.
  • Asmussen, S. (2000). Ruin Probabilities. World Scientific, Singapore.
  • Athreya, K. B., McDonald, D. and Ney, P. (1978). Limit theorems for semi-Markov processes and renewal theory for Markov chains. Ann. Probab. 6 788–797.
  • Billingsley, P. (1968). Convergence of Probability Measures. Wiley, New York.
  • Bougerol, P. (1988). Théorémes limite pour les systémes linéaires à coefficients markoviens. Probab. Theory Related Fields 78 193–221.
  • Bougerol, P. and Lacroix, J. (1985). Product of Random Matrices with Application to Schrödinger Operators. Birkhäuser, Basel.
  • Bougerol, P. and Picard, N. (1992). Stationarity of GARCH processes and of some non-negative time series. J. Econometrics 52 115–128.
  • Carlsson, H. (1983). Remainder term estimates of the renewal function. Ann. Probab. 11 143–157.
  • Carlsson, H. and Wainger, S. (1982). An asymptotic series expansion of the multidimensional renewal measure. Comput. Math. 47 355–364.
  • Fuh, C. D. (1997). Corrected diffusion approximations for ruin probabilities in Markov random walk. Adv. in Appl. Probab. 29 695–712.
  • Fuh, C. D. and Lai, T. L. (1998). Wald's equations, first passage times and moments of ladder variables in Markov random walks. J. Appl. Probab. 35 566–580.
  • Fuh, C. D. and Lai, T. L. (2001). Asymptotic expansions in multidimensional Markov renewal theory and first passage times for Markov random walks. Adv. in Appl. Probab. 33 652–673.
  • Fuh, C. D. and Zhang, C. H. (2000). Poisson equation, moment inequalities and quick convergence for Markov random walks. Stochastic Process. Appl. 87 53–67.
  • Ganelius, T. (1971). Tauberian Remainder Theorems. Lecture Notes in Math. 232. Springer, Berlin.
  • Gelfand, I. M. and Shilov, G. E. (1964). Generalized Functions 1. Academic Press, New York.
  • Heyde, C. C. (1964). Two probability theorems and their applications to some first passage problems. J. Austral. Math. Soc. 4 214–222.
  • Hipp, C. (1985). Asymptotic expansions in the central limit theorem for compound and Markov processes. Z. Wahrsch. Verw. Gebiete 69 361–385.
  • Jensen, J. L. (1987). A note on asymptotic expansions for Markov chains using operator theory. Adv. in Appl. Math. 8 377–392.
  • Jensen, J. L. (1989). Asymptotic expansions for strongly mixing Harris recurrent Markov chains. Scand. J. Statist. 16 47–63.
  • Kartashov, N. V. (1980). A uniform asymptotic renewal theorem. Theory Probab. Appl. 25 589–592.
  • Kartashov, N. V. (1996). Strong Stable Markov Chains. VSP, Utrecht and TBiMC Scientific Publishers, Kiev.
  • Kesten, H. (1973). Random difference equations and renewal theory for products of random matrices. Acta Math. 131 311–349.
  • Kesten, H. (1974). Renewal theory for functionals of a Markov chain with general state space. Ann. Probab. 2 355–386.
  • Kovalenko, I. N., Kuznetsov, N. Yu. and Shurenkov, V. M. (1996). Models of Random Processes. A Handbook for Mathematicians and Engineers. CRC Press, Boca Raton, FL.
  • Lai, T. L. (1976). Uniform Tauberian theorems and their applications to renewal theory and first passage problems. Ann. Probab. 4 628–643.
  • Lai, T. L. and Siegmund, D. (1983). Fixed accuracy estimation of an autoregressive parameter. Ann. Statist. 11 478–485.
  • Lorden, G. (1970). On excess over the boundary. Ann. Math. Statist. 41 520–527.
  • Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, New York.
  • Malinovskii, V. K. (1986). Asymptotic expansions in the central limit theorem for recurrent Markov renewal processes. Theory Probab. Appl. 31 523–526.
  • Malinovskii, V. K. (1987). Limit theorems for Harris–Markov chains, I. Theory Probab. Appl. 31 269–285.
  • Nagaev, S. V. (1957). Some limit theorems for stationary Markov chains. Theory Probab. Appl. 2 378–406.
  • Nagaev, S. V. (1961). More exact statements of limit theorems for homogeneous Markov chains. Theory Probab. Appl. 6 62–81.
  • Ney, P. and Nummelin, E. (1987). Markov additive processes I. Eigenvalue properties and limit theorems. Ann. Probab. 15 561–592.
  • Nummelin, E. (1984). General Irreducible Markov Chains and Non-negative Operators. Cambridge Univ. Press.
  • Pergamenshchikov, S. M. and Shiryaev, A. N. (1993). Sequential estimation of the parameters of a stochastic difference equation with random coefficients. Theory Probab. Appl. 37 449–470.
  • Riesz, F. and Sz-Nagy, B. (1955). Functional Analysis. Ungar, New York.
  • Schwartz, L. (1966). Théorie des Distributions, 2nd ed. Hermann, Paris.
  • Shurenkov, V. M. (1984). On the theory of Markov renewal. Theory Probab. Appl. 9 247–265.
  • Shurenkov, V. M. (1989). Ergodic Markov Processes. Nauka, Moscow.
  • Siegmund, D. (1979). Corrected diffusion approximations in certain random walk problems. Adv. in Appl. Probab. 11 701–719.
  • Siegmund, D. (1985). Sequential Analysis. Springer, New York.
  • Siegmund, D. and Yuh, Y. S. (1982). Brownian approximations to first passage probabilities. Z. Wahrsch. Verw. Gebiete 59 239–248.
  • Silvestrov, D. S. (1978). The renewal theorem in a series scheme I. Theory Probab. Math. Statist. 18 144–161.
  • Silvestrov, D. S. (1979). The renewal theorem in a series scheme II. Theory Probab. Math. Statist. 20 97–116.
  • Silvestrov, D. S. (1994). Coupling for Markov renewal processes and the rate of convergence in ergodic theorems for processes with semi-Markov switchings. Acta Appl. Math. 34 109–124.
  • Silvestrov, D. S. (1995). Exponential asymptotic for perturbed renewal equations. Theory Probab. Math. Statist. 52 153–162.
  • Silvestrov, D. S. and Brusilovskii, I. L. (1990). An invariance principle for sums stationary connected random variables satisfying a uniform strong mixing condition with weight. Theory Probab. Math. Statist. 40 117–127.
  • Statulevicius, V. (1969). Limit theorems for sums of random variables that are connected in a Markov chain. I, II. Litovsk. Mat. Sb. 9 345–362; 10 635–672.
  • Stone, C. (1965a). On characteristic functions and renewal theory. Trans. Amer. Math. Soc. 120 327–342.
  • Stone, C. (1965b). On moment generating functions and renewal theory. Ann. Math. Statist. 36 298–301.
  • Strichartz, R. (1994). A Guide to Distribution Theory and Fourier Transforms. CRC Press, Boca Raton, FL.
  • Zhang, C. H. (1989). A renewal theory with varying drift. Ann. Probab. 17 723–736.