The Annals of Applied Probability

Spectral theory and limit theorems for geometrically ergodic Markov processes

I. Kontoyiannis and S. P. Meyn

Full-text: Open access

Abstract

Consider the partial sums $\{S_t\}$ of a real-valued functional $F(\Phi(t))$ of a Markov chain $\{\Phi(t)\}$ with values in a general state space. Assuming only that the Markov chain is geometrically ergodic and that the functional $F$ is bounded, the following conclusions are obtained:

Spectral theory. Well-behaved solutions $\cf$ can be constructed for the "multiplicative Poisson equation" $(e^{\alpha F}P)\cf=\lambda\cf$, where P is the transition kernel of the Markov chain and $\alpha\in\Co$ is a constant. The function $\cf$ is an eigenfunction, with corresponding eigenvalue $\lambda$, for the kernel $(e^{\alpha F}P)=e^{\alpha F(x)}P(x,dy)$.

A "multiplicative" mean ergodic theorem. For all complex $\alpha$ in a neighborhood of the origin, the normalized mean of $\exp(\alpha S_t)$ (and not the logarithm of the mean) converges to $\cf$ exponentially, where $\cf$ is a solution of the multiplicative Poisson equation.

Edgeworth expansions. Rates are obtained for the convergence of the distribution function of the normalized partial sums $S_t$ to the standard Gaussian distribution. The first term in this expansion is of order $(1/\sqrt{t})$ and it depends on the initial condition of the Markov chain through the solution $\haF$ of the associated Poisson equation (and not the solution $\cf$ of the multiplicative Poisson equation).

Large deviations. The partial sums are shown to satisfy a large deviations principle in a neighborhood of the mean. This result, proved under geometric ergodicity alone, cannot in general be extended to the whole real line.

Exact large deviations asymptotics. Rates of convergence are obtained for the large deviations estimates above. The polynomial preexponent is of order $(1/\sqrt{t})$ and its coefficient depends on the initial condition of the Markov chain through the solution $\cf$ of the multiplicative Poisson equation.

Extensions of these results to continuous-time Markov processes are also given.

Article information

Source
Ann. Appl. Probab. Volume 13, Number 1 (2003), 304-362.

Dates
First available in Project Euclid: 16 January 2003

Permanent link to this document
http://projecteuclid.org/euclid.aoap/1042765670

Digital Object Identifier
doi:10.1214/aoap/1042765670

Mathematical Reviews number (MathSciNet)
MR1952001

Zentralblatt MATH identifier
1016.60066

Subjects
Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces) 60F10: Large deviations 37L40: Invariant measures 60J25: Continuous-time Markov processes on general state spaces 41A36: Approximation by positive operators

Keywords
Markov process large deviations Edgeworth expansions positive harmonic function Poisson equation

Citation

Kontoyiannis, I.; Meyn, S. P. Spectral theory and limit theorems for geometrically ergodic Markov processes. Ann. Appl. Probab. 13 (2003), no. 1, 304--362. doi:10.1214/aoap/1042765670. http://projecteuclid.org/euclid.aoap/1042765670.


Export citation

References

  • [1] BAHADUR, R. R. and RANGA RAO, R. (1960). On deviations of the sample mean. Ann. Math. Statist. 31 1015-1027.
  • [2] BALAJI, S. and MEy N, S. P. (2000). Multiplicative ergodicity and large deviations for an irreducible Markov chain. Stochastic Process. Appl. 90 123-144.
  • [3] BEZHAEVA, Z. I. and OSELEDETS, V. I. (1996). On the variance of sums for functions of a stationary Markov process. Teor. Veroy atnost. i Primenen. 41 633-639.
  • [4] BOLTHAUSEN, E., DEUSCHEL, J.-D. and TAMURA, Y. (1995). Laplace approximations for large deviations of nonreversible Markov processes. The nondegenerate case. Ann. Probab. 23 236-267.
  • [5] BRy C, W. and DEMBO, A. (1996). Large deviations and strong mixing. Ann. Inst. H. Poincaré Probab. Statist. 32 549-569.
  • [6] BUDHIRAJA, A. and DUPUIS, P. (2001). Large deviations for the empirical measure of reflecting Brownian motion and related constrained processes in R+. Preprint.
  • [7] CHAGANTY, N. R. and SETHURAMAN, J. (1993). Strong large deviation and local limit theorems. Ann. Probab. 21 1671-1690.
  • [8] DATTA, S. and MCCORMICK, W. P. (1993). On the first-order Edgeworth expansion for a Markov chain. J. Multivariate Anal. 44 345-359.
  • [9] DE ACOSTA, A. (1990). Large deviations for empirical measures of Markov chains. J. Theoret. Probab. 3 395-431.
  • [10] DE ACOSTA, A. (1997). Moderate deviations for empirical measures of Markov chains: Lower bounds. Ann. Probab. 25 259-284.
  • [11] DE ACOSTA, A. and CHEN, X. (1998). Moderate deviations for empirical measures of Markov chains: Upper bounds. J. Theoret. Probab. 11 1075-1110.
  • [12] DE ACOSTA, A. and NEY, P. (1998). Large deviation lower bounds for arbitrary additive functionals of a Markov chain. Ann. Probab. 26 1660-1682.
  • [13] DEMBO, A. and ZEITOUNI, O. (1998). Large Deviations Techniques and Applications, 2nd ed. Springer, New York.
  • [14] DEUSCHEL, J. D. and STROOCK, D. W. (1989). Large Deviations. Academic, Boston.
  • [15] DOWN, D., MEy N, S. P. and TWEEDIE, R. L. (1995). Exponential and uniform ergodicity of Markov processes. Ann. Probab. 23 1671-1691.
  • [16] ELLIS, R. S. (1988). Large deviations for the empirical measure of a Markov chain with an application to the multivariate empirical measure. Ann. Probab. 16 1496-1508.
  • [17] FELLER, W. (1971). An Introduction to Probability Theory and Its Applications 2, 2nd ed. Wiley, New York.
  • [18] FENG, J. (1999). Martingale problems for large deviations of Markov processes. Stochastic Process. Appl. 81 165-212.
  • [19] FENG, J. and KURTZ, T. G. (2000). Large deviations for stochastic processes. Preprint.
  • [20] FLEMING, W. H. (1978). Exit probabilities and optimal stochastic control. Appl. Math. Optim. 4 329-346.
  • [21] FLEMING, W. H. (1997). Some results and problems in risk sensitive stochastic control. Mat. Appl. Comput. 16 99-115.
  • [22] FLEMING, W. H. and SHEU, S.-J. (1997). Asy mptotics for the principal eigenvalue and eigenfunction of a nearly first-order operator with large potential. Ann. Probab. 25 1953- 1994.
  • [23] GLy NN, P. W. and MEy N, S. P. (1996). A Liapunov bound for solutions of the Poisson equation. Ann. Probab. 24 916-931.
  • [24] HALL, P. (1982). Rates of Convergence in the Central Limit Theorem. Pitman, Boston.
  • [25] HORDIJK, A. and SPIEKSMA, F. (1992). On ergodicity and recurrence properties of a Markov chain with an application to an open Jackson network. Adv. in Appl. Probab. 24 343-376.
  • [26] HUANG, J., KONTOy IANNIS, I. and MEy N, S. P. (2002). The ODE method and spectral theory of Markov operators. Stochastic Theory and Control. Lecture Notes in Control and Inform. Sci. 205-221. Springer, New York.
  • [27] HUISINGA, W., MEy N, S. P. and SCHUETTE, C. (2001). Phase transitions and metastability in Markovian and molecular sy stems. Preprint.
  • [28] ISCOE, I., NEY, P. and NUMMELIN, E. (1985). Large deviations of uniformly recurrent Markov additive processes. Adv. in Appl. Math. 6 373-412.
  • [29] JENSEN, J. L. (1987). A note on asy mptotic expansions for Markov chains using operator theory. Adv. in Appl. Math. 8 377-392.
  • [30] JENSEN, J. L. (1991). Saddlepoint expansions for sums of Markov dependent variables on a continuous state space. Probab. Theory Related Fields 89 181-199.
  • [31] KARTASHOV, N. V. (1985). Criteria for uniform ergodicity and strong stability of Markov chains with a common phase space. Theory Probab. Appl. 30 71-89.
  • [32] KARTASHOV, N. V. (1985). Inequalities in theorems of ergodicity and stability for Markov chains with a common phase space. Theory Probab. Appl. 30 247-259.
  • [33] KONTOy IANNIS, I. and MEy N, S. P. (2002). Large deviation asy mptotics and the spectral theory of multiplicatively regular Markov processes. Preprint.
  • [34] KUMAR, P. R. and MEy N, S. P. (1996). Duality and linear programs for stability and performance analysis queueing networks and scheduling policies. IEEE Trans. Automat. Control 41 4-17.
  • [35] MEy N, S. P. and TWEEDIE, R. L. (1993). Stability of Markovian processes III: Foster- Ly apunov criteria for continuous time processes. Adv. in Appl. Probab. 25 518-548.
  • [36] MEy N, S. P. and TWEEDIE, R. L. (1993). Generalized resolvents and Harris recurrence of Markov processes. In Doeblin and Modern Probability 227-250. Amer. Math. Soc., Providence, RI.
  • [37] MEy N, S. P. and TWEEDIE, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.
  • [38] MEy N, S. P. and TWEEDIE, R. L. (1994). Computable bounds for geometric convergence rates of Markov chains. Ann. Appl. Probab. 4 981-1011.
  • [39] MILLER, H. D. (1961). A convexivity property in the theory of random variables defined on a finite Markov chain. Ann. Math. Statist. 32 1260-1270.
  • [40] NAGAEV, S. V. (1957). Some limit theorems for stationary Markov chains. Theory Probab. Appl. 2 378-406.
  • [41] NAGAEV, S. V. (1961). More exact limit theorems for homogeneous Markov chains. Theory Probab. Appl. 6 62-81.
  • [42] NEY, P. and NUMMELIN, E. (1987). Markov additive processes I. Eigenvalue properties and limit theorems. Ann. Probab. 15 561-592.
  • [43] NEY, P. and NUMMELIN, E. (1987). Markov additive processes II. Large deviations. Ann. Probab. 15 593-609.
  • [44] NIEMI, S. and NUMMELIN, E. (1986). On nonsingular renewal kernels with an application to a semigroup of transition kernels. Stochastic Process. Appl. 22 177-202.
  • [45] NUMMELIN, E. (1984). General Irreducible Markov Chains and Nonnegative Operators. Cambridge Univ. Press.
  • [46] PETROV, V. V. (1995). Limit Theorems of Probability Theory. Clarendon, New York.
  • [47] PINSKY, R. G. (1995). Positive Harmonic Functions and Diffusion. Cambridge Univ. Press.
  • [48] RIESZ, F. and SZ.-NAGY, B. (1955). Functional Analy sis. Ungar, New York.
  • [49] SCHWERER, E. (1997). A linear programming approach to the steady-state analysis of Markov processes. Ph.D. thesis, Stanford Univ.
  • [50] SHURENKOV, V. M. (1984). On Markov renewal theory. Teor. Veroy atnost. i Primenen. 29 248-263.
  • [51] STROOCK, D. W. (1984). An Introduction to the Theory of Large Deviations. Springer, New York.
  • [52] VARADHAN, S. R. S. (1984). Large Deviations and Applications. SIAM, Philadelphia.
  • [53] VARADHAN, S. R. S. (1985). Large deviations and applications. Exposition. Math. 3 251-272.
  • [54] WEIS, L. (1984). Approximation by weakly compact operators in L1. Math. Nachr. 118 321- 326.
  • [55] WILLIAMS, R. J. (1985). Reflected Brownian motion in a wedge: Semimartingale property. Z. Wahrsch. Verw. Gebiete 69 161-176.
  • [56] WU, L. (2000). Some notes on large deviations of Markov processes. Acta Math. Sinica 16 369-394.
  • BOX F, 182 GEORGE STREET
  • PROVIDENCE, RHODE ISLAND 02912 E-MAIL: yiannis@dam.brown.edu www.dam.brown.edu/people/yiannis/ DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING AND THE COORDINATED SCIENCES LABORATORY UNIVERSITY OF ILLINOIS
  • URBANA, ILLINOIS 61801 E-MAIL: s-mey n@uiuc.edu www.black.csl.uiuc.edu/ mey n/