Bayesian analysis for reversible Markov chains

Abstract

We introduce a natural conjugate prior for the transition matrix of a reversible Markov chain. This allows estimation and testing. The prior arises from random walk with reinforcement in the same way the Dirichlet prior arises from Pólya’s urn. We give closed form normalizing constants, a simple method of simulation from the posterior and a characterization along the lines of W. E. Johnson’s characterization of the Dirichlet prior.