The Annals of Applied Probability

An interruptible algorithm for perfect sampling via Markov chains

James Allen Fill

Full-text: Open access

Abstract

For a large class of examples arising in statistical physics known as attractive spin systems (e.g., the Ising model), one seeks to sample from a probability distribution $\pi$ on an enormously large state space, but elementary sampling is ruled out by the infeasibility of calculating an appropriate normalizing constant. The same difficulty arises in computer science problems where one seeks to sample randomly from a large finite distributive lattice whose precise size cannot be ascertained in any reasonable amount of time.

The Markov chain Monte Carlo (MCMC) approximate sampling approach to such a problem is to construct and run "for a long time" a Markov chain with long-run distribution $\pi$. But determining how long is long enough to get a good approximation can be both analytically and empirically difficult.

Recently, Propp and Wilson have devised an ingenious and efficient algorithm to use the same Markov chains to produce perfect (i.e., exact) samples from $\pi$. However, the running time of their algorithm is an unbounded random variable whose order of magnitude is typically unknown a priori and which is not independent of the state sampled, so a naive user with limited patience who aborts a long run of the algorithm will introduce bias.

We present a new algorithm which (1) again uses the same Markov chains to produce perfect samples from $\pi$, but is based on a different idea (namely, acceptance/rejection sampling); and (2) eliminates user-impatience bias. Like the Propp-Wilson algorithm, the new algorithm applies to a general class of suitably monotone chains, and also (with modification) to "anti-monotone" chains. When the chain is reversible, naive implementation of the algorithm uses fewer transitions but more space than Propp-Wilson. When fine-tuned and applied with the aid of a typical pseudorandom number generator to an attractive spin system on n sites using a random site updating Gibbs sampler whose mixing time $\tau$ is polynomial in n, the algorithm runs in time of the same order (bound) as Propp-Wilson [expectation $O(\tau \log n)$] and uses only logarithmically more space [expectation $O(n \log n)$, vs.$O9n)$ for Propp-Wilson].

Article information

Source
Ann. Appl. Probab. Volume 8, Number 1 (1998), 131-162.

Dates
First available in Project Euclid: 29 July 2002

Permanent link to this document
http://projecteuclid.org/euclid.aoap/1027961037

Digital Object Identifier
doi:10.1214/aoap/1027961037

Mathematical Reviews number (MathSciNet)
MR1620346

Zentralblatt MATH identifier
0939.60084

Subjects
Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 68U20: Simulation [See also 65Cxx] 60G40: Stopping times; optimal stopping problems; gambling theory [See also 62L15, 91A60] 62D05: Sampling theory, sample surveys

Keywords
Markov chain Monte Carlo perfect simulation rejection sampling monotone chain attractive spin system Ising model Gibbs sampler separation strong stationary time duality partially ordered set

Citation

Fill, James Allen. An interruptible algorithm for perfect sampling via Markov chains. Ann. Appl. Probab. 8 (1998), no. 1, 131--162. doi:10.1214/aoap/1027961037. http://projecteuclid.org/euclid.aoap/1027961037.


Export citation

References

  • [1] Aldous, D. and Diaconis, P. (1986). Shuffling cards and stopping times. Amer. Math. Monthly 93 333-348.
  • [2] Aldous, D. and Diaconis, P. (1987). Strong uniform times and finite random walks. Adv. in Appl. Math. 8 69-97.
  • [3] Aldous, D. and Fill, J. A. (1998). Reversible Markov Chains and Random Walks on Graphs. Unpublished manuscript. First draft of manuscript available via http://www.stat. berkeley.edu/users/aldous.
  • [4] Asmussen, S., Gly nn, P. W. and Thorisson, H. (1992). Stationary detection in the initial transient problem. ACM Trans. Modelling and Comput. Sim. 2 130-157.
  • [5] Besag, J. E. (1986). On the statistical analysis of dirty pictures. J. Roy. Statist. Soc. Ser. B 48 259-302.
  • [6] Besag, J. E. and Clifford, P. (1989). Generalized Monte Carlo significance tests. Biometrika 76 633-642.
  • [7] Besag, J. E. and Clifford, P. (1991). Sequential Monte Carlo p-values. Biometrika 78 301- 304.
  • [8] Besag, J., Green, P., Higdon, D. and Mengersen, K. (1995). Bayesian computation and stochastic sy stems. Statist. Sci. 10 3-66.
  • [9] Borovkov, A. A. and Foss, S. G. (1992). Stochastically recursive sequences and their generalizations. Siberian Adv. Math. 2 16-81.
  • [10] Borovkov, A. A. and Foss, S. G. (1994). Two ergodicity criteria for stochastically recursive sequences. Acta Appl. Math. 34 125-134.
  • [11] Cowen, L. J. (1996). Personal communication.
  • [12] Diaconis, P. (1988). Group Representations in Probability and Statistics. IMS, Hay ward, CA.
  • [13] Diaconis, P. and Fill, J. A. (1990). Strong stationary times via a new form of duality. Ann. Probab. 18 1483-1522.
  • [14] Diaconis, P. and Saloff-Coste, L. (1995). What do we know about the Metropolis algorithm? In Proceedings of the Twenty-Seventh Annual ACM Sy mposium on the Theory of Computing 112-129.
  • [15] Diaconis, P. and Saloff-Coste, L. (1995). Geometry and randomness. Unpublished lecture notes.
  • [16] Fill, J. A. (1998). The move-to-front rule: a case study for two exact sampling algorithms. Probab. Eng. Info. Sci. To appear.
  • [17] Fill, J. A. and Machida, M. (1998). Stochastic monotonicity and realizable monotonicity. Unpublished notes.
  • [18] Fill, J. A. and Machida, M. (1997). Duality relations for Markov chains on partially ordered state spaces. Unpublished notes.
  • [19] Foss, S. G. (1983). On ergodicity conditions in multi-server queues. Siberian Math. J. 24 168-175.
  • [20] Foss, S., Tweedie, R. L. and Corcoran, J. (1997). Simulating the invariant measures of Markov chains using backward coupling at regeneration times. Preprint.
  • [21] Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analy sis and Machine Intelligence PAMI-6(6).
  • [22] H¨aggstr ¨om, O. and Nelander, K. (1997). Exact sampling from anti-monotone sy stems. Preprint.
  • [23] H¨aggstr ¨om, O., van Lieshout, M. N. M. and Møller, J. (1996). Characterisation results and Markov chain Monte Carlo algorithms including exact simulation for some spatial point processes. Preprint.
  • [24] Johnson, V. E. (1996). Studying convergence of Markov chain Monte Carlo algorithms using coupled sample paths. J. Amer. Statist. Assoc. 91 154-166.
  • [25] Kamae, T., Krengel, U. and O'Brien, G. L. (1977). Stochastic inequalities on partially ordered state spaces. Ann. Probab. 5 899-912.
  • [26] Kendall, W. S. (1996). Perfect simulation for the area-interaction point process. In Probability Perspective (L. Accardi and C. C. Hey de, eds.). World Scientific Press, Singapore. To appear.
  • [27] Kendall, W. S. (1996). On some weighted Boolean models. In Advances in Theory and Application of Random Sets (D. Jeulin, ed.) 105-120. World Scientific Press, Singapore.
  • [28] Liggett, T. (1985). Interacting Particle Sy stems. Springer, New York.
  • [29] Lov´asz, L. and Winkler, P. (1995). Exact mixing in an unknown Markov chain. Electron. J. Combin. 2 R15.
  • [30] Lund, R. B., Wilson, D. B., Foss, S. and Tweedie, R. L. (1997). Exact and approximate simulation of the invariant measures of Markov chains. Preprint.
  • [31] Propp, J. G. and Wilson, D. B. (1996). Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Structures and Algorithms 9 223-252.
  • [32] Ross, S. (1994). A First Course in Probability, 4th ed. Macmillan, New York.
  • [33] Sinclair, A. (1993). Algorithms for Random Generation and Counting: A Markov Chain Approach. Birkh¨auser, Boston.
  • [34] Sperner, E. (1928). Ein Satz ¨uber Untermengen einer endlichen Menge. Math. Z. 27 544- 548.
  • [35] Stanley, R. P. (1986). Enumerative Combinatorics 1. Wadsworth & Brooks/Cole, Monterey, CA.
  • [36] Strassen, V. (1965). The existence of probability measures with given marginals. Ann. Math. Statist. 36 423-439.
  • [37] Wilson, D. B. and Propp, J. G. (1997). How to get a perfectly random sample from a generic Markov chain and generate a random spanning tree of a directed graph. J. Algorithms. To appear.
  • [38] Wilson, D. B. (1995). Personal communication.