The Annals of Statistics

Bayesian analysis of variable-order, reversible Markov chains

Sergio Bacallado

Full-text: Open access


We define a conjugate prior for the reversible Markov chain of order r. The prior arises from a partially exchangeable reinforced random walk, in the same way that the Beta distribution arises from the exchangeable Polyá urn. An extension to variable-order Markov chains is also derived. We show the utility of this prior in testing the order and estimating the parameters of a reversible Markov model.

Article information

Ann. Statist., Volume 39, Number 2 (2011), 838-864.

First available in Project Euclid: 9 March 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62M02: Markov processes: hypothesis testing
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures

Reversibility reinforced random walks variable-order Markov chains Bayesian analysis conjugate priors


Bacallado, Sergio. Bayesian analysis of variable-order, reversible Markov chains. Ann. Statist. 39 (2011), no. 2, 838--864. doi:10.1214/10-AOS857.

Export citation


  • [1] Aldous, D. and Fill, J. Reversible Markov chains and random walks on graphs. Unpublished manuscript.
  • [2] Bacallado, S. (2010). Supplement to “Bayesian analysis of variable-order, reversible Markov chains.” DOI:10.1214/10-AOS857SUPP.
  • [3] Bacallado, S., Chodera, J. D. and Pande, V. (2009). Bayesian comparison of Markov models of molecular dynamics with detailed balance constraint. J. Chem. Phys. 131 045106.
  • [4] Begleiter, R., El-Yaniv, R. and Yona, G. (2004). On prediction using variable order Markov models. J. Artificial Intelligence Res. 22 385–421.
  • [5] Buchete, N. and Hummer, G. (2008). Coarse master equations for peptide folding dynamics. J. Phys. Chem. B 112 6057–6069.
  • [6] Bühlmann, P. (2000). Model selection for variable length Markov chains and tuning the context algorithm. Ann. Inst. Statist. Math. 52 287–315.
  • [7] Bühlmann, P. and Wyner, A. J. (1999). Variable length Markov chains. Ann. Statist. 27 480–513.
  • [8] Diaconis, P. and Freedman, D. (1980). de Finetti theorem for Markov chains. Ann. Probab. 8 115–130.
  • [9] Diaconis, P. (1988). Recent progress on de Finetti notions of exchangeability. In Bayesian Statistics 3 ( J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 111–125. Oxford Univ. Press, New York.
  • [10] Diaconis, P. and Rolles, S. (2006). Bayesian analysis for reversible Markov chains. Ann. Statist. 34 1270–1292.
  • [11] Kass, R. and Raftery, A. (1995). Bayes factors. J. Amer. Statist. Assoc. 90 773–795.
  • [12] Keane, M. and Rolles, S. (2000). Edge-reinforced random walk on finite graphs. In Infinite Dimensional Stochastic Analysis 217–234. R. Neth. Acad. Arts Sci., Amsterdam.
  • [13] Kelly, F. P. (1979). Reversibility and Stochastic Networks. Wiley, Chichester.
  • [14] MacQueen, J. (1981). Circuit processes. Ann. Probab. 9 604–610.
  • [15] Rissanen, J. (1983). A universal data compression system. IEEE Trans. Inform. Theory 29 656–664.
  • [16] Rolles, S. (2003). How edge-reinforced random walk arises naturally. Probab. Theory Related Fields 126 243–260.
  • [17] Weinberger, M., Rissanen, J. and Feder, M. (1995). A universal finite memory source. IEEE Trans. Inform. Theory 41 643–652.
  • [18] Zabell, S. (1982). W. E. Johnson’s sufficientness postulate. Ann. Statist. 10 1091–1099.

Supplemental materials

  • Supplementary material: Law of a variable-order, reinforced random walk. We provide a closed form expression for this law as a function of transition counts and suggest how it could be useful.