Electronic Communications in Probability

A Hoeffding inequality for Markov chains

Shravas Rao

Full-text: Open access

Abstract

We prove deviation bounds for the random variable $\sum _{i=1}^{n} f_i(Y_i)$ in which $\{Y_i\}_{i=1}^{\infty }$ is a Markov chain with stationary distribution and state space $[N]$, and $f_i: [N] \rightarrow [-a_i, a_i]$. Our bound improves upon previously known bounds in that the dependence is on $\sqrt{a_1^2+\cdots +a_n^2} $ rather than $\max _{i}\{a_i\}\sqrt{n} .$ We also prove deviation bounds for certain types of sums of vector–valued random variables obtained from a Markov chain in a similar manner. One application includes bounding the expected value of the Schatten $\infty $-norm of a random matrix whose entries are obtained from a Markov chain.

Article information

Source
Electron. Commun. Probab., Volume 24 (2019), paper no. 14, 11 pp.

Dates
Received: 13 September 2018
Accepted: 17 February 2019
First available in Project Euclid: 21 March 2019

Permanent link to this document
https://projecteuclid.org/euclid.ecp/1553133703

Digital Object Identifier
doi:10.1214/19-ECP219

Subjects
Primary: 60F10: Large deviations

Keywords
Hoeffding bound Markov chain generic chaining random matrices

Rights
Creative Commons Attribution 4.0 International License.

Citation

Rao, Shravas. A Hoeffding inequality for Markov chains. Electron. Commun. Probab. 24 (2019), paper no. 14, 11 pp. doi:10.1214/19-ECP219. https://projecteuclid.org/euclid.ecp/1553133703


Export citation

References

  • [1] Rudolf Ahlswede and Andreas Winter, Strong converse for identification via quantum channels, IEEE Trans. Inform. Theory 48 (2002), no. 3, 569–579.
  • [2] Afonso S. Bandeira and Ramon van Handel, Sharp nonasymptotic bounds on the norm of random matrices with independent entries, Ann. Probab. 44 (2016), no. 4, 2479–2506.
  • [3] Kai-Min Chung, Henry Lam, Zhenming Liu, and Michael Mitzenmacher, Chernoff-Hoeffding bounds for Markov chains: Generalized and simplified, STACS, 2012, arXiv:1201.0559, pp. 124–135.
  • [4] I. H. Dinwoodie, A probability inequality for the occupation measure of a reversible Markov chain, Ann. Appl. Probab. 5 (1995), no. 1, 37–43.
  • [5] Jianqing Fan, Bai Jiang, and Qiang Sun, Hoeffding’s lemma for Markov chains and its applications to statistical learning, 2018.
  • [6] Ankit Garg, Yin Tat Lee, Zhao Song, and Nikhil Srivastava, A matrix expander Chernoff bound, 2017.
  • [7] David Gillman, A Chernoff bound for random walks on expander graphs, SIAM J. Comput. 27 (1998), no. 4, 1203–1220.
  • [8] Jan Hazła and Thomas Holenstein, Upper tail estimates with combinatorial proofs, STACS, 2015, arXiv:1405.2349, pp. 392–405.
  • [9] Alexander D. Healy, Randomness-efficient sampling within ${\rm NC}^1$, Comput. Complexity 17 (2008), no. 1, 3–37.
  • [10] Wassily Hoeffding, Probability inequalities for sums of bounded random variables, J. Amer. Statist. Assoc. 58 (1963), 13–30.
  • [11] Nabil Kahale, Large deviation bounds for Markov chains, Combin. Probab. Comput. 6 (1997), no. 4, 465–474.
  • [12] Carlos A. León and François Perron, Optimal Hoeffding bounds for discrete reversible Markov chains, Ann. Appl. Probab. 14 (2004), no. 2, 958–970.
  • [13] Pascal Lezaud, Chernoff-type bound for finite Markov chains, Ann. Appl. Probab. 8 (1998), no. 3, 849–867.
  • [14] Assaf Naor, On the Banach-space-valued Azuma inequality and small-set isoperimetry of Alon-Roichman graphs, Combin. Probab. Comput. 21 (2012), no. 4, 623–634.
  • [15] Assaf Naor, Shravas Rao, and Oded Regev, On the rate of convergence of the vector-valued ergodic theorem for Markov chains with a spectral gap, 2017, In preparation.
  • [16] Daniel Paulin, Concentration inequalities for Markov chains by Marton couplings and spectral methods, Electron. J. Probab. 20 (2015), 32 pp.
  • [17] Shravas Rao and Oded Regev, A sharp tail bound for the expander random sampler, 2017.
  • [18] Walter Rudin, Functional analysis, second ed., International Series in Pure and Applied Mathematics, McGraw-Hill, Inc., New York, 1991.
  • [19] Michel Talagrand, Regularity of Gaussian processes, Acta Math. 159 (1987), no. 1-2, 99–149.
  • [20] Michel Talagrand, Upper and lower bounds for stochastic processes, Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd Series. A Series of Modern Surveys in Mathematics], vol. 60, Springer, Heidelberg, 2014, Modern methods and classical problems.
  • [21] Joel A. Tropp, An introduction to matrix concentration inequalities, Found. Trends Mach. Learn. 8 (2015), no. 1-2, 1–230.
  • [22] Roy Wagner, Tail estimates for sums of variables sampled by a random walk, Comb. Probab. Comput. 17 (2008), no. 2, 307–316, arXiv:math/0608740.