The Annals of Applied Probability

Analysis of a nonreversible Markov chain sampler

Persi Diaconis, Susan Holmes, and Radford M. Neal

Full-text: Open access

Abstract

We analyze the convergence to stationarity of a simple nonreversible Markov chain that serves as a model for several nonreversible Markov chain sampling methods that are used in practice. Our theoretical and numerical results show that nonreversibility can indeed lead to improvements over the diffusive behavior of simple Markov chain sampling schemes. The analysis uses both probabilistic techniques and an explicit diagonalization.

Article information

Source
Ann. Appl. Probab., Volume 10, Number 3 (2000), 726-752.

Dates
First available in Project Euclid: 22 April 2002

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1019487508

Digital Object Identifier
doi:10.1214/aoap/1019487508

Mathematical Reviews number (MathSciNet)
MR1789978

Zentralblatt MATH identifier
1083.60516

Subjects
Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 65U05

Keywords
Nonreversible Markov chain Markov chain Monte Carlo Metropolis algorithm

Citation

Diaconis, Persi; Holmes, Susan; Neal, Radford M. Analysis of a nonreversible Markov chain sampler. Ann. Appl. Probab. 10 (2000), no. 3, 726--752. doi:10.1214/aoap/1019487508. https://projecteuclid.org/euclid.aoap/1019487508


Export citation

References

  • [1] Adler, S. L. (1981). Over-relaxation method for the Monte Carlo evaluation of the partition function for multiquadratic actions. Phys. Rev. D 23 2901-2904.
  • [2] Bassiri, F. (1997). Random walks on finite groups of multiplicity two. Ph.D. dissertation, Harvard Univ.
  • [3] Binder, K. (1979). Monte Carlo Methods in Statistical Physics. Springer, Berlin.
  • [4] Conway, J., Sloane, N. and Wilks, A. (1989). Gray codes for reflection groups. Graphs Combin. 5 315-325.
  • [5] Chen, F., Lov´asz, L. and Pak, I. (1999). Lifting Markov chains to speed up mixing. In Proceedings of the 31st Annual ACM Symposium on Theory of Computing 275-281. ACM Press, New York.
  • [6] Chung, F., Graham, R. and Yau, S. T. (1996). On sampling with Markov chains. Random Structures Algorithms 9 55-77.
  • [7] Diaconis, P. (1988). Group Representations in Probability and Statistics. IMS, Hayward, CA.
  • [8] Diaconis, P. and Efron, B. (1987). Probabilistic-geometric theorems arising from the analysis of contingency tables. In Contributions to the Theory and Application of Statistics: A Volume in Honor of Herbert Solomon (A. E. Gelfand ed.) 103-125. Academic Press, Boston.
  • [9] Diaconis, P. and Fill, J. (1990). Strong stationary times via a new form of duality. Ann. Probab. 18 1483-1522.
  • [10] Diaconis, P. and Gangolli, A. (1996). Rectangular arrays with fixed margins. In Finite Markov Chain Renaissance 15-42. Springer, New York.
  • [11] Diaconis, P. and Holmes, S. (1994). Gray codes for randomization procedures. Statist. Comput. 4 207-302.
  • [12] Diaconis, P. and Saloff-Coste, L. (1994). Moderate growth and random walk on finite groups. Geom. Funct. Anal. 4 1-36.
  • [13] Diaconis, P. and Saloff-Coste, L. (1998). What do we know about the Metropolis algorithm? J. Comput. System Sci. 57 20-36.
  • [14] Diaconis, P. and Sturmfels, B. (1998). Algebraic algorithms for sampling from conditional distributions. Ann. Statist. 26 363-397.
  • [15] Duane, S., Kennedy, A. D., Pendleton, B. J. and Roweth, D. (1987). Hybrid Monte Carlo. Phys. Lett. B 195 216-222.
  • [16] Gelfand, A. E. and Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. J. Amer. Statist. Assoc. 85 398-409.
  • [17] Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Machine Intelligence 6 721-741.
  • [18] Gustafson, P. (1997). Large hierarchical Bayesian analysis of multivariate survival data. Biometrics 53 230-242.
  • [19] Gustafson, P. (1998). A guided walk Metropolis algorithm. Statist. Comput. 8 357-364.
  • [20] Hildebrand, M. (1997). Rates of convergence for a non-reversible Markov chain sampler. Preprint. Available from http://math.albany.edu:8000/ martinhi/.
  • [21] Horowitz, A. M. (1991). A generalized guided Monte Carlo algorithm. Phys. Lett. B 268 247-252.
  • [22] Lindvall, T. (1992). Lectures on the Coupling Method. Wiley, New York.
  • [23] Kennedy, A. D. (1990). The theory of hybrid stochastic algorithms. In Probabilistic Methods in Quantum Field Theory and Quantum Gravity (P. H. Damgaard, H. H ¨uffel and A. Rosenblum, eds.). Plenum, New York.
  • [24] Liu, J. (1996). Metropolized independent sampling with comparisons to rejection sampling and importance sampling. Statist. Comput. 6 113-119.
  • [25] Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E.
  • (1953). Equation of state calculations by fast computing machines. J. Chem. Phys. 21 1087-1092.
  • [26] Mira, A. and Geyer, C. J. (1999). Ordering Monte Carlo Markov chains. Technical Report 632, School of Statistics, Univ. Minnesota.
  • [27] Neal, R. M. (1993). Probabilistic inference using Markov chain Monte Carlo methods. Technical Report CRG-TR-93-1, Dept. Computer Science, Univ. Toronto. Available from http://www.cs.utoronto.ca/ radford/.
  • [28] Neal, R. M. (1996). Bayesian Learning for Neural Networks. Lecture Notes in Statist. 118 Springer, New York.
  • [29] Neal, R. M. (1998). Suppressing random walks in Markov chain Monte Carlo using ordered overrelaxation. In Learning in Graphical Models (M. I. Jordan, ed.) 205-225. Kluwer, Dordrecht.
  • [30] Roberts, G. O. and Sahu, S. K. (1997). Updating schemes, correlation structure, blocking and parameterisation for the Gibbs sampler. J. Roy. Statist. Soc. B 59 291-317.
  • [31] Sinclair, A. (1993). Algorithms for Random Generation and Counting: A Markov Chain Approach. Birkh¨auser, Boston.
  • [32] Smith, A. F. M. and Roberts, G. O. (1993). Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods. J. Roy. Statist. Soc. 55 3-23.
  • [33] Tierney, L. (1994). Markov Chains for exploring posterior distributions. Ann. Statist. 22 1701-1762.
  • [34] Toussaint, D. (1989). Introduction to algorithms for Monte Carlo simulations and their application to QCD. Comput. Phys. Comm. 56 69-92.