The Annals of Statistics

Bayesian analysis for reversible Markov chains

Persi Diaconis and Silke W. W. Rolles

Full-text: Open access


We introduce a natural conjugate prior for the transition matrix of a reversible Markov chain. This allows estimation and testing. The prior arises from random walk with reinforcement in the same way the Dirichlet prior arises from Pólya’s urn. We give closed form normalizing constants, a simple method of simulation from the posterior and a characterization along the lines of W. E. Johnson’s characterization of the Dirichlet prior.

Article information

Ann. Statist., Volume 34, Number 3 (2006), 1270-1292.

First available in Project Euclid: 10 July 2006

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62M02: Markov processes: hypothesis testing
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures

Bayesian analysis reversible Markov chains conjugate priors hypothesis testing


Diaconis, Persi; Rolles, Silke W. W. Bayesian analysis for reversible Markov chains. Ann. Statist. 34 (2006), no. 3, 1270--1292. doi:10.1214/009053606000000290.

Export citation


  • Chan, C. S., Robbins, D. P. and Yuen, D. S. (2000). On the volume of a certain polytope. Experiment. Math. 9 91--99.
  • Dalal, S. R. and Hall, W. J. (1983). Approximating priors by mixtures of natural conjugate priors. J. Roy. Statist. Soc. Ser. B 45 278--286.
  • Diaconis, P. (1988). Recent progress on de Finetti's notions of exchangeability. In Bayesian Statistics 3 (J.-M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 111--125. Oxford Univ. Press, New York.
  • Diaconis, P. and Freedman, D. (1980). de Finetti's theorem for Markov chains. Ann. Probab. 8 115--130.
  • Diaconis, P. and Ylvisaker, D. (1985). Quantifying prior opinion (with discussion). In Bayesian Statistics 2 (J.-M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 133--156. North-Holland, Amsterdam.
  • Fortini, S., Ladelli, L., Petris, G. and Regazzini, E. (2002). On mixtures of distributions of Markov chains. Stochastic Process. Appl. 100 147--165.
  • Freedman, D. A. (1962). Mixtures of Markov processes. Ann. Math. Statist. 33 114--118.
  • Giblin, P. J. (1981). Graphs, Surfaces and Homology. An Introduction to Algebraic Topology, 2nd ed. Chapman and Hall, London.
  • Good, I. J. (1968). The Estimation of Probabilities. An Essay on Modern Bayesian Methods. MIT Press, Cambridge, MA.
  • Good, I. J. and Crook, J. F. (1987). The robustness and sensitivity of the mixed-Dirichlet Bayesian test for ``independence'' in contingency tables. Ann. Statist. 15 670--693.
  • Höglund, T. (1974). Central limit theorems and statistical inference for finite Markov chains. Z. Wahrsch. Verw. Gebiete 29 123--151.
  • Keane, M. S. (1990). Solution to problem 288. Statist. Neerlandica 44 95--100.
  • Keane, M. S. and Rolles, S. W. W. (2000). Edge-reinforced random walk on finite graphs. In Infinite Dimensional Stochastic Analysis (Ph. Clément, F. den Hollander, J. van Neerven and B. de Pagter, eds.) 217--234. R. Neth. Acad. Arts Sci., Amsterdam.
  • Maurer, S. B. (1976). Matrix generalizations of some theorems on trees, cycles and cocycles in graphs. SIAM J. Appl. Math. 30 143--148.
  • Pemantle, R. (1988). Phase transition in reinforced random walk and RWRE on trees. Ann. Probab. 16 1229--1241.
  • Rolles, S. W. W. (2003). How edge-reinforced random walk arises naturally. Probab. Theory Related Fields 126 243--260.
  • Stanley, R. P. (1978). Generating functions. In Studies in Combinatorics (G.-C. Rota, ed.) 100--141. Math. Assoc. Amer., Washington.
  • Zabell, S. L. (1982). W. E. Johnson's ``sufficientness'' postulate. Ann. Statist. 10 1090--1099.
  • Zabell, S. L. (1995). Characterizing Markov exchangeable sequences. J. Theoret. Probab. 8 175--178.