Electronic Journal of Probability

Concentration inequalities for Markov chains by Marton couplings and spectral methods

Daniel Paulin

Full-text: Open access

Abstract

We prove a version of McDiarmid’s bounded differences inequality for Markov chains, with constants proportional to the mixing time of the chain. We also show variance bounds and Bernstein-type inequalities for empirical averages of Markov chains. In the case of non-reversible chains, we introduce a new quantity called the “pseudo spectral gap”, and show that it plays a similar role for non-reversible chains as the spectral gap plays for reversible chains. Our techniques for proving these results are based on a coupling construction of Katalin Marton, and on spectral techniques due to Pascal Lezaud. The pseudo spectral gap generalises the multiplicative reversiblication approach of Jim Fill.

Article information

Source
Electron. J. Probab., Volume 20 (2015), paper no. 79, 32 pp.

Dates
Accepted: 27 July 2015
First available in Project Euclid: 4 June 2016

Permanent link to this document
https://projecteuclid.org/euclid.ejp/1465067185

Digital Object Identifier
doi:10.1214/EJP.v20-4039

Mathematical Reviews number (MathSciNet)
MR3383563

Zentralblatt MATH identifier
1342.60121

Subjects
Primary: 60E15: Inequalities; stochastic orderings 60J05: Discrete-time Markov processes on general state spaces 60J10: Markov chains (discrete-time Markov processes on discrete state spaces) 28A35: Measures and integrals in product spaces
Secondary: 05C81: Random walks on graphs 68Q87: Probability in computer science (algorithm analysis, random structures, phase transitions, etc.) [See also 68W20, 68W40]

Keywords
Concentration inequalities Markov chain mixing time spectral gap coupling

Rights
This work is licensed under aCreative Commons Attribution 3.0 License.

Citation

Paulin, Daniel. Concentration inequalities for Markov chains by Marton couplings and spectral methods. Electron. J. Probab. 20 (2015), paper no. 79, 32 pp. doi:10.1214/EJP.v20-4039. https://projecteuclid.org/euclid.ejp/1465067185


Export citation

References

  • Adamczak, Radosław. A tail inequality for suprema of unbounded empirical processes with applications to Markov chains. Electron. J. Probab. 13 (2008), no. 34, 1000–1034.
  • Adamczak, Radosław and Bednorz, Witold (2012). Exponential concentration inequalities for additive functionals of Markov chains. arXiv:1201.3569.
  • Boucheron, Stéphane, Lugosi, Gábor and Massart, Pascal (2013). Concentration Inequalities: A Nonasymptotic Theory of Independence. OUP Oxford.
  • Chazottes, Jean-Rene; Redig, Frank. Concentration inequalities for Markov processes via coupling. Electron. J. Probab. 14 (2009), no. 40, 1162–1180.
  • Chazottes, J.-R.; Collet, P.; Kulske, C.; Redig, F. Concentration inequalities for random fields via coupling. Probab. Theory Related Fields 137 (2007), no. 1-2, 201–225.
  • Devroye, Luc; Lugosi, Gábor. Combinatorial methods in density estimation. Springer Series in Statistics. Springer-Verlag, New York, 2001. xii+208 pp. ISBN: 0-387-95117-2
  • Diaconis, Persi; Holmes, Susan; Montgomery, Richard. Dynamical bias in the coin toss. SIAM Rev. 49 (2007), no. 2, 211–235.
  • Djellout, H.; Guillin, A.; Wu, L. Transportation cost-information inequalities and applications to random dynamical systems and diffusions. Ann. Probab. 32 (2004), no. 3B, 2702–2732.
  • Doeblin, Wolfang (1938). Exposé de la théorie des chaines simples constantes de Markova un nombre fini d'états. Mathématique de l'Union Interbalkanique 78–80.
  • Douc, Randal; Moulines, Eric; Olsson, Jimmy; van Handel, Ramon. Consistency of the maximum likelihood estimator for general hidden Markov models. Ann. Statist. 39 (2011), no. 1, 474–513.
  • Dvoretzky, A.; Kiefer, J.; Wolfowitz, J. Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator. Ann. Math. Statist. 27 (1956), 642–669.
  • Fiebig, Doris. Mixing properties of a class of Bernoulli-processes. Trans. Amer. Math. Soc. 338 (1993), no. 1, 479–493.
  • Fill, James Allen. Eigenvalue bounds on convergence to stationarity for nonreversible Markov chains, with an application to the exclusion process. Ann. Appl. Probab. 1 (1991), no. 1, 62–87.
  • Gillman, David. A Chernoff bound for random walks on expander graphs. SIAM J. Comput. 27 (1998), no. 4, 1203–1220.
  • Glynn, Peter W.; Ormoneit, Dirk. Hoeffding's inequality for uniformly ergodic Markov chains. Statist. Probab. Lett. 56 (2002), no. 2, 143–146.
  • Goldstein, Sheldon. Maximal coupling. Z. Wahrsch. Verw. Gebiete 46 (1978/79), no. 2, 193–204.
  • Gyori, Benjamin and Paulin, Daniel (2014). Non-asymptotic confidence intervals for MCMC in practice. arXiv:1212.2016
  • Hu, ShuLan. Transportation inequalities for hidden Markov chains and applications. Sci. China Math. 54 (2011), no. 5, 1027–1042.
  • Janson, Svante. Large deviations for sums of partly dependent random variables. Random Structures Algorithms 24 (2004), no. 3, 234–248.
  • Kontorovich,L.L. (2006). Measure concentration of hidden Markov processes. math/0608064.
  • Kontorovich, Leonid. Measure concentration of strongly mixing processes with applications. Thesis (Ph.D.) - Carnegie Mellon University. ProQuest LLC, Ann Arbor, MI, 2007. 79 pp. ISBN: 978-0549-19078-3 http://www.cs.bgu.ac.il/~karyeh/thesis.pdf
  • Kontorovich, Leonid; Ramanan, Kavita. Concentration inequalities for dependent random variables via the martingale method. Ann. Probab. 36 (2008), no. 6, 2126–2158.
  • Kontorovich, A. Obtaining measure concentration from Markov contraction. Markov Process. Related Fields 18 (2012), no. 4, 613–638.
  • Ledoux, Michel. The concentration of measure phenomenon. Mathematical Surveys and Monographs, 89. American Mathematical Society, Providence, RI, 2001. x+181 pp. ISBN: 0-8218-2864-9
  • Leon, Carlos A.; Perron, Francois. Optimal Hoeffding bounds for discrete reversible Markov chains. Ann. Appl. Probab. 14 (2004), no. 2, 958–970.
  • Levin, David A.; Peres, Yuval; Wilmer, Elizabeth L. Markov chains and mixing times. With a chapter by James G. Propp and David B. Wilson. American Mathematical Society, Providence, RI, 2009. xviii+371 pp. ISBN: 978-0-8218-4739-8
  • Lezaud, Pascal. Chernoff-type bound for finite Markov chains. Ann. Appl. Probab. 8 (1998), no. 3, 849–867.
  • Lezaud,P.P. (1998b). Etude quantitative des cha^ines de Markov par perturbation de leur noyau. Thèse doctorat mathématiques appliquées de l'Université Paul Sabatier de Toulouse, Available at http://pom.tls.cena.fr/papers/thesis/these_lezaud.pdf.
  • Lezaud, Pascal. Chernoff and Berry-Esseen inequalities for Markov processes. ESAIM Probab. Statist. 5 (2001), 183–201.
  • Lindvall, Torgny. Lectures on the coupling method. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. A Wiley-Interscience Publication. John Wiley & Sons, Inc., New York, 1992. xiv+257 pp. ISBN: 0-471-54025-0
  • Lubetzky, E. and Sly,A. (2009). Cutoff for the Ising model on the lattice. Inventiones Mathematicae 1–37.
  • Marton, Katalin. Bounding d-distance by informational divergence: a method to prove measure concentration. Ann. Probab. 24 (1996), no. 2, 857–866.
  • Marton, Katalin. A measure concentration inequality for contracting Markov chains. Geom. Funct. Anal. 6 (1996), no. 3, 556–571.
  • Marton, Katalin. Measure concentration for a class of random processes. Probab. Theory Related Fields 110 (1998), no. 3, 427–439.
  • Marton, Katalin. Measure concentration and strong mixing. Studia Sci. Math. Hungar. 40 (2003), no. 1-2, 95–113.
  • Massart, Pascal. The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality. Ann. Probab. 18 (1990), no. 3, 1269–1283.
  • Merlevede, Florence; Peligrad, Magda; Rio, Emmanuel. A Bernstein type inequality and moderate deviations for weakly dependent sequences. Probab. Theory Related Fields 151 (2011), no. 3-4, 435–474.
  • Ollivier, Yann. Ricci curvature of Markov chains on metric spaces. J. Funct. Anal. 256 (2009), no. 3, 810–864.
  • Paulin, Daniel. (2014a) Mixing and Concentration by Ricci Curvature. 2014. arXiv:1404.2802.
  • Paulin, Daniel. (2014b) Concentration inequalities for dependent random variables. Thesis (Ph.D.), National University of Singapore.
  • Rio, Emmanuel. Inegalites de Hoeffding pour les fonctions lipschitziennes de suites dependantes. (French) [Hoeffding inequalities for Lipschitz functions of dependent sequences] C. R. Acad. Sci. Paris Ser. I Math. 330 (2000), no. 10, 905–908.
  • Roberts, Gareth O.; Rosenthal, Jeffrey S. General state space Markov chains and MCMC algorithms. Probab. Surv. 1 (2004), 20–71.
  • Rosenthal, Jeffrey S. Faithful couplings of Markov chains: now equals forever. Adv. in Appl. Math. 18 (1997), no. 3, 372–381.
  • Samson, Paul-Marie. Concentration of measure inequalities for Markov chains and Phi-mixing processes. Ann. Probab. 28 (2000), no. 1, 416–461.
  • Wintenberger, Olivier (2012). Weak transport inequalities and applications to exponential inequalities and oracle inequalities. arXiv:1207.4951