The Annals of Applied Probability

Markov chain decomposition for convergence rate analysis

Neal Madras and Dana Randall

Full-text: Open access

Abstract

In this paper we develop tools for analyzing the rate at which a reversible Markov chain converges to stationarity. Our techniques are useful when the Markov chain can be decomposed into pieces which are themselves easier to analyze. The main theorems relate the spectral gap of the original Markov chains to the spectral gaps of the pieces. In the first case the pieces are restrictions of the Markov chain to subsets of the state space; the second case treats a Metropolis--Hastings chain whose equilibrium distribution is a weighted average of equilibrium distributions of other Metropolis--Hastings chains on the same state space.

Article information

Source
Ann. Appl. Probab., Volume 12, Number 2 (2002), 581-606.

Dates
First available in Project Euclid: 17 July 2002

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1026915617

Digital Object Identifier
doi:10.1214/aoap/1026915617

Mathematical Reviews number (MathSciNet)
MR1910641

Zentralblatt MATH identifier
1017.60080

Subjects
Primary: 60J05: Discrete-time Markov processes on general state spaces
Secondary: 65C05: Monte Carlo methods 68Q25: Analysis of algorithms and problem complexity [See also 68W40]

Keywords
Markov chain Monte Carlo spectral gap Metropolis-Hastings algorithm simulated tempering decomposition

Citation

Madras, Neal; Randall, Dana. Markov chain decomposition for convergence rate analysis. Ann. Appl. Probab. 12 (2002), no. 2, 581--606. doi:10.1214/aoap/1026915617. https://projecteuclid.org/euclid.aoap/1026915617


Export citation

References

  • BORGS, C., CHAy ES, J. T., FRIEZE, A., KIM, J. H., TETALI, P., VIGODA, E. and VU, V. H.
  • (1999). Torpid mixing of some MCMC algorithms in statistical physics. In Proceedings of the 40th IEEE Sy mposium on Foundations of Computer Science 218-229. IEEE, New York.
  • BRODER, A. Z. (1986). How hard is it to marry at random? (On the approximation of the permanent). In Proceedings of the 18th ACM Sy mposium on Theory of Computing 50-58. ACM, New
  • York. [Erratum in Broder, A. Z. (1988). Proceedings of the 20th ACM Sy mposium on Theory of Computing 551.]
  • BUBLEY, R. and Dy ER, M. (1997). Path coupling, Dobrushin uniqueness, and approximate counting. Technical Report 97.04, School of Computer Science, University of Leeds.
  • CARACCIOLO, S., PELISSETTO, A. and SOKAL, A. D. (1992). Two remarks on simulated tempering. Unpublished manuscript.
  • COOPER, C., Dy ER, M., FRIEZE, A. and RUE, R. (2000). Mixing properties of the Swendsen-Wang process on the complete graph and narrow grids. J. Math. Phy s. 41 1499-1527.
  • DIACONIS, P. and SALOFF-COSTE, L. (1993a). Comparison techniques for random walks on finite groups. Ann. Probab. 21 2131-2156.
  • DIACONIS, P. and SALOFF-COSTE, L. (1993b). Comparison theorems for reversible Markov chains. Ann. Appl. Probab. 3 696-730.
  • DIACONIS, P. and STROOCK, D. (1993). Geometric bounds for Markov chains. Ann. Appl. Probab. 1 36-61.
  • GEy ER, C. J. (1991). Markov chain Monte Carlo maximum likelihood. In Computing Science and Statistics: Proceedings of the 23rd Sy mposium on the Interface (E. M. Keramidas, ed.) 156-163. Interface Foundation, Fairfax Station, VA.
  • GEy ER, C. J. and THOMPSON, E. A. (1995). Annealing Markov chain Monte Carlo with applications to ancestral inference. J. Amer. Statist. Assoc. 90 909-920.
  • HALMOS, P. R. (1982). A Hilbert Space Problem Book, 2nd ed. Springer, New York.
  • JERRUM, M. R. and SINCLAIR, A. J. (1989). Approximating the permanent. SIAM J. Comput. 18 1149-1178.
  • LUBY, M., RANDALL, D. and SINCLAIR, A. J. (1995). Markov chain algorithms for planar lattice structures. In Proc. 36th IEEE Sy mposium on Foundations of Computer Science 150-159.
  • IEEE, New York.
  • LUBY, M., RANDALL, D. and SINCLAIR, A. J. (2001). Markov chains for planar lattice structures. SIAM J. Comput. 31 167-192.
  • LUBY, M. and VIGODA, E. (1997). Approximately counting up to four. In Proc. 29th ACM Sy mposium on Theory of Computing 682-687. ACM, New York.
  • LUBY, M. and VIGODA, E. (1999). Fast convergence of the Glauber dy namics for sampling independent sets. Random Structures Algorithms 15 229-241.
  • MADRAS, N. (1998). Umbrella sampling and simulated tempering. In Numerical Methods for Poly meric Sy stems (S. G. Whittington, ed.) 19-32. Springer, New York.
  • MADRAS, N. and PICCIONI, M. (1999). Importance sampling for families of distributions. Ann. Appl. Probab. 9 1202-1225.
  • MADRAS, N. and RANDALL, D. (1996). Factoring graphs to bound mixing rates. In Proc. 37th IEEE Sy mposium on Foundations of Computer Science 194-203. IEEE, New York.
  • MADRAS, N. and SLADE, G. (1993). The Self-Avoiding Walk. Birkhäuser, Boston.
  • MARINARI, E. and PARISI, G. (1992). Simulated tempering: a new Monte Carlo scheme. Europhy s. Lett. 19 451-458.
  • ORLANDINI, E. (1998). Monte Carlo study of poly mer sy stems by Multiple Markov Chain method. In Numerical Methods for Poly meric Sy stems (S. G. Whittington, ed.) 33-58. Springer, New York.
  • RANDALL, D. and TETALI, P. (1998). Analy zing Glauber dy namics by comparison of Markov chains. In Proc. LATIN '98: Theoretical Informatics. Lecture Notes in Computer Sci. 1380 292-304. Springer, New York.
  • ROBERTS, G. O. and ROSENTHAL, J. S. (1997). Geometric ergodicity and hy brid Markov chains. Electron. Comm. Probab. 2 13-25.
  • ROBERTS, G. O. and TWEEDIE, R. L. (2001). Geometric L2 and L1 convergence are equivalent for reversible Markov chains. J. Appl. Probab. 38A 37-41.
  • SINCLAIR, A. J. (1993). Randomized Algorithms for Counting and Generating Combinatorial Structures. Birkhäuser, Boston.
  • SOKAL, A. D. and THOMAS, L. E. (1989). Exponential convergence to equilibrium for a class of random-walk models. J. Statist. Phy s. 54 797-828.
  • TIERNEY, L. (1998). A note on Metropolis-Hastings kernels for general state spaces. Ann. Appl. Probab. 8 1-9.
  • TORRIE, G. M. and VALLEAU, J. P. (1977). Nonphysical sampling distributions in Monte Carlo free energy estimation: Umbrella sampling. J. Comput. Phy s. 23 187-199.
  • WELSH, D. J. A. (1993). Complexity: Knots, Colourings, and Counting. Cambridge Univ. Press.
  • YOSIDA, K. (1980). Functional Analy sis, 6th ed. Springer, Berlin.
  • ZHENG, Z. (1999). Analy sis of swapping and tempering Monte Carlo algorithms. Ph.D. Thesis, York Univ.
  • TORONTO, ONTARIO M3J 1P3 CANADA E-MAIL: madras@mathstat.yorku.ca COLLEGE OF COMPUTING AND SCHOOL OF MATHEMATICS GEORGIA INSTITUTE OF TECHNOLOGY ATLANTA GEORGIA 30332-0280 E-MAIL: randall@cc.gatech.edu