Bernoulli

  • Bernoulli
  • Volume 20, Number 4 (2014), 1698-1716.

Convergence rate and concentration inequalities for Gibbs sampling in high dimension

Neng-Yi Wang and Liming Wu

Full-text: Open access

Abstract

The objective of this paper is to study the Gibbs sampling for computing the mean of observable in very high dimension – a powerful Markov chain Monte Carlo method. Under the Dobrushin’s uniqueness condition, we establish some explicit and sharp estimate of the exponential convergence rate and prove some Gaussian concentration inequalities for the empirical mean.

Article information

Source
Bernoulli, Volume 20, Number 4 (2014), 1698-1716.

Dates
First available in Project Euclid: 19 September 2014

Permanent link to this document
https://projecteuclid.org/euclid.bj/1411134441

Digital Object Identifier
doi:10.3150/13-BEJ537

Mathematical Reviews number (MathSciNet)
MR3263086

Zentralblatt MATH identifier
06368414

Keywords
concentration inequality coupling method Dobrushin’s uniqueness condition Gibbs measure Markov chain Monte Carlo

Citation

Wang, Neng-Yi; Wu, Liming. Convergence rate and concentration inequalities for Gibbs sampling in high dimension. Bernoulli 20 (2014), no. 4, 1698--1716. doi:10.3150/13-BEJ537. https://projecteuclid.org/euclid.bj/1411134441


Export citation

References

  • [1] Bobkov, S.G. and Götze, F. (1999). Exponential integrability and transportation cost related to logarithmic Sobolev inequalities. J. Funct. Anal. 163 1–28.
  • [2] Diaconis, P., Khare, K. and Saloff-Coste, L. (2008). Gibbs sampling, exponential families and orthogonal polynomials. Statist. Sci. 23 151–178.
  • [3] Diaconis, P., Khare, K. and Saloff-Coste, L. (2010). Gibbs sampling, conjugate priors and coupling. Sankhya A 72 136–169.
  • [4] Djellout, H., Guillin, A. and Wu, L. (2004). Transportation cost-information inequalities and applications to random dynamical systems and diffusions. Ann. Probab. 32 2702–2732.
  • [5] Dobrushin, R.L. (1968). The description of a random field by means of conditional probabilities and condition of its regularity. Theory Probab. Appl. 13 197–224.
  • [6] Dobrushin, R.L. (1970). Prescribing a system of random variables by conditional distributions. Theory Probab. Appl. 15 458–486.
  • [7] Dobrushin, R.L. and Shlosman, S.B. (1985). Completely analytical Gibbs fields. In Statistical Physics and Dynamical Systems (Köszeg, 1984). Progress in Probability 10 371–403. Boston, MA: Birkhäuser.
  • [8] Doucet, A., de Freitas, N. and Gordon, N. (2001). Sequential Monte Carlo Methods in Practice. Statistics for Engineering and Information Science. New York: Springer.
  • [9] Gozlan, N. and Léonard, C. (2007). A large deviation approach to some transportation cost inequalities. Probab. Theory Related Fields 139 235–283.
  • [10] Gozlan, N. and Léonard, C. (2010). Transport inequalities. A survey. Markov Process. Related Fields 16 635–736.
  • [11] Hairer, M. and Mattingly, J.C. (2011). Yet another look at Harris’ ergodic theorem for Markov chains. In Seminar on Stochastic Analysis, Random Fields and Applications VI. Progress in Probability 63 109–117. Basel: Birkhäuser.
  • [12] Joulin, A. and Ollivier, Y. (2010). Curvature, concentration and error estimates for Markov chain Monte Carlo. Ann. Probab. 38 2418–2442.
  • [13] Ledoux, M. (1999). Concentration of measure and logarithmic Sobolev inequalities. In Séminaire de Probabilités, XXXIII. Lecture Notes in Math. 1709 120–216. Berlin: Springer.
  • [14] Ledoux, M. (2001). The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89. Providence, RI: Amer. Math. Soc.
  • [15] Martinelli, F. (1999). Lectures on Glauber dynamics for discrete spin models. In Lectures on Probability Theory and Statistics (Saint-Flour, 1997). Lecture Notes in Math. 1717 93–191. Berlin: Springer.
  • [16] Martinelli, F. and Olivieri, E. (1994). Approach to equilibrium of Glauber dynamics in the one phase region. I. The attractive case. Comm. Math. Phys. 161 447–486.
  • [17] Marton, K. (1996). Bounding $\overline{d}$-distance by informational divergence: A method to prove measure concentration. Ann. Probab. 24 857–866.
  • [18] Marton, K. (1996). A measure concentration inequality for contracting Markov chains. Geom. Funct. Anal. 6 556–571.
  • [19] Marton, K. (2003). Measure concentration and strong mixing. Studia Sci. Math. Hungar. 40 95–113.
  • [20] Marton, K. (2004). Measure concentration for Euclidean distance in the case of dependent random variables. Ann. Probab. 32 2526–2544.
  • [21] Meyn, S.P. and Tweedie, R.L. (1993). Markov Chains and Stochastic Stability. Communications and Control Engineering Series. London: Springer.
  • [22] Paulin, D. (2012). Concentration inequalities for Markov chains by Marton coupling. Preprint.
  • [23] Rio, E. (2000). Inégalités de Hoeffding pour les fonctions lipschitziennes de suites dépendantes. C. R. Acad. Sci. Paris Sér. I Math. 330 905–908.
  • [24] Roberts, G.O. and Rosenthal, J.S. (2004). General state space Markov chains and MCMC algorithms. Probab. Surv. 1 20–71.
  • [25] Rosenthal, J.S. (1995). Minorization conditions and convergence rates for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 90 558–566.
  • [26] Rosenthal, J.S. (1996). Analysis of the Gibbs sampler for a model related to James–Stein estimations. Statist. Comput. 6 269–275.
  • [27] Rosenthal, J.S. (2002). Quantitative convergence rates of Markov chains: A simple account. Electron. Commun. Probab. 7 123–128 (electronic).
  • [28] Samson, P.M. (2000). Concentration of measure inequalities for Markov chains and $\Phi$-mixing processes. Ann. Probab. 28 416–461.
  • [29] Talagrand, M. (1996). Transportation cost for Gaussian and other product measures. Geom. Funct. Anal. 6 587–600.
  • [30] Villani, C. (2003). Topics in Optimal Transportation. Graduate Studies in Mathematics 58. Providence, RI: Amer. Math. Soc.
  • [31] Winkler, G. (1995). Image Analysis, Random Fields and Dynamic Monte Carlo Methods: A Mathematical Introduction. Applications of Mathematics (New York) 27. Berlin: Springer.
  • [32] Wintenberger, O. (2012). Weak transport inequalities and applications to exponential inequalities and oracle inequalities. Preprint.
  • [33] Wu, L. (2006). Poincaré and transportation inequalities for Gibbs measures under the Dobrushin uniqueness condition. Ann. Probab. 34 1960–1989.
  • [34] Zegarliński, B. (1992). Dobrushin uniqueness theorem and logarithmic Sobolev inequalities. J. Funct. Anal. 105 77–111.