The Annals of Applied Probability

Chernoff-type bound for finite Markov chains

Pascal Lezaud

Full-text: Open access

Abstract

This paper develops bounds on the distribution function of the empirical mean for irreducible finite-state Markov chains. One approach, explored by Gillman, reduces this problem to bounding the largest eigenvalue of a perturbation of the transition matrix for the Markov chain. By using estimates on eigenvalues given in Kato's book Perturbation Theory for Linear Operators, we simplify the proof of Gillman and extend it to nonreversible finite-state Markov chains and continuous time. We also set out another method, directly applicable to some general ergodic Markov kernels having a spectral gap.

Article information

Source
Ann. Appl. Probab. Volume 8, Number 3 (1998), 849-867.

Dates
First available in Project Euclid: 9 August 2002

Permanent link to this document
http://projecteuclid.org/euclid.aoap/1028903453

Mathematical Reviews number (MathSciNet)
MR1627795

Digital Object Identifier
doi:10.1214/aoap/1028903453

Zentralblatt MATH identifier
0938.60027

Subjects
Primary: 60F10: Large deviations

Keywords
Markov chain Chernoff bound eigenvalues perturbation theory

Citation

Lezaud, Pascal. Chernoff-type bound for finite Markov chains. Ann. Appl. Probab. 8 (1998), no. 3, 849--867. doi:10.1214/aoap/1028903453. http://projecteuclid.org/euclid.aoap/1028903453.


Export citation

References

  • Aldous, D. (1987). On the Markov chain simulation method for uniform combinatorial distributions and simulated annealing. Prob. Engrg. Inform. Sci. 1 33-46. Aldous, D. and Fill, J. Reversible Markov chains and random walks on graphs. Unpublished manuscript.
  • Bahadur, R. R. and Ranga Rao, R. (1960). On deviations of the sample mean. Ann. Math. Statist. 31 1015-1033.
  • Bennett, G. (1962). Probability inequalities for sums of independent random variables. J. Amer. Statist. Assoc. 57 33-45.
  • Chernoff, H. (1952). A measure of asy mptotic efficiency for tests of a hy pothesis based on the sum of observations. Ann. Math. Statist. 23 493-507.
  • Cram´er, H. (1938). Sur un nouveau th´eor eme de la th´eorie des probabilit´es. Actualit´es Sci. Indust. 736.
  • Dembo, A. and Zeitouni, O. (1993). Large Deviations Techniques and Applications. Jones and Bartlett, Boston.
  • Diaconis, P. and Saloff-Coste, L. (1993). Comparison theorems for reversible Markov chains. Ann. Appl. Probab. 3 696-730.
  • Diaconis, P. and Saloff-Coste, L. (1995). What do we know about the Metropolis algorithm? Unpublished manuscript. Diaconis, P. and Saloff-Coste, L. (1996a). Logarithmic Sobolev inequalities and finite Markov chains. Ann. Appl. Probab. 6 695-750. Diaconis, P. and Saloff-Coste, L. (1996b). Nash inequalities for finite Markov chains. J. Appl. Probab. 9 459-510.
  • Dinwoodie, I. (1995). A probability inequality for the occupation measure of a reversible Markov chain. Ann. Appl. Probab. 5 37-43.
  • Fill, J. (1991). Eigenvalue bounds on convergence to stationarity for nonreversible Markov chains, with an application to the exclusion process. Ann. Appl. Probab. 1 62-87.
  • Gillman, D. (1993). Hidden Markov chains: rates of convergence and the complexity of inference. Ph.D. dissertation, MIT.
  • Kato, T. (1966). Perturbation Theory for Linear Operators. Springer, New York.
  • Kolmogorov, A. (1929). ¨Uber das Gesetz des iterieten Logaritmus. Math. Ann. 99 309-319.
  • Mann, B. (1996). Berry-Esseen central limit theorem for Markov chains. Ph.D. dissertation, Harvard Univ.
  • Marshall, A. W. and Olkin, I. (1979). Inequalities: Theory of Majorization and Its Applications. Academic Press, New York.
  • Nagaev, S. V. (1957). Some limit theorems for stationary Markov chains. Theory Probab. Appl. 2 378-406.
  • Nagaev, S. V. (1961). More exact statements of limits theorems for homogeneous Markov chains. Theory Probab. Appl. 6 62-81.
  • Stout, W. F. (1974). Almost Sure Convergence. Academic Press, New York.
  • Trotter, H. F. (1959). On the product of semi-groups of operators. Proc. Amer. Math. Soc. 10 545-551.