Electronic Journal of Statistics

A comparison theorem for data augmentation algorithms with applications

Hee Min Choi and James P. Hobert

Full-text: Open access

Abstract

The data augmentation (DA) algorithm is considered a useful Markov chain Monte Carlo algorithm that sometimes suffers from slow convergence. It is often possible to convert a DA algorithm into a sandwich algorithm that is computationally equivalent to the DA algorithm, but converges much faster. Theoretically, the reversible Markov chain that drives the sandwich algorithm is at least as good as the corresponding DA chain in terms of performance in the central limit theorem and in the operator norm sense. In this paper, we use the sandwich machinery to compare two DA algorithms. In particular, we provide conditions under which one DA chain can be represented as a sandwich version of the other. Our results are used to extend Hobert and Marchev’s (2008) results on the Haar PX-DA algorithm and to improve the collapsing theorem of Liu et al. (1994) and Liu (1994). We also illustrate our results using Brownlee’s (1965) stack loss data.

Article information

Source
Electron. J. Statist., Volume 10, Number 1 (2016), 308-329.

Dates
Received: July 2015
First available in Project Euclid: 17 February 2016

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1455715964

Digital Object Identifier
doi:10.1214/16-EJS1106

Mathematical Reviews number (MathSciNet)
MR3466184

Zentralblatt MATH identifier
1338.60189

Subjects
Primary: 60J27: Continuous-time Markov processes on discrete state spaces
Secondary: 62F15: Bayesian inference

Keywords
Data augmentation algorithm sandwich algorithm central limit theorem convergence rate operator norm

Citation

Choi, Hee Min; Hobert, James P. A comparison theorem for data augmentation algorithms with applications. Electron. J. Statist. 10 (2016), no. 1, 308--329. doi:10.1214/16-EJS1106. https://projecteuclid.org/euclid.ejs/1455715964


Export citation

References

  • Brownlee, K. A. (1965)., Statistical Theory and Methodology in Science and Engineering. 2nd ed. Wiley, New York.
  • Choi, H. M. (2014)., Convergence Analysis of Gibbs Samplers for Bayesian Regression Models. Ph.D. thesis, University of Florida.
  • Choi, H. M. and Hobert, J. P. (2013). Analysis of MCMC algorithms for Bayesian linear regression with Laplace errors., Journal of Multivariate Analysis, 117 32–40.
  • Conway, J. B. (1990)., A Course in Functional Analysis. 2nd ed. Springer, New York.
  • Faden, A. M. (1985). The existence of regular conditional probabilities: Necessary and sufficient conditions., The Annals of Probability, 13 288–298.
  • Gebelein, H. (1941). Das statistische Problem der Korrelation als Variations und Eigenwertproblem und sein Zusammenhang mit der Ausgleichsrechnung., Zeitschrift für Angewandte Mathematik und Mechanik, 21 364–379.
  • Hobert, J. P. and Marchev, D. (2008). A theoretical comparison of the data augmentation, marginal augmentation and PX-DA algorithms., The Annals of Statistics, 36 532–554.
  • Hobert, J. P. and Román, J. C. (2011). Comment: “To center or not to center: That is not the question – An Ancillarity-Sufficiency Interweaving Strategy (ASIS) for boosting MCMC efficiency”., Journal of Computational and Graphical Statistics, 20 571–580.
  • Jones, G. L., Haran, M., Caffo, B. S. and Neath, R. (2006). Fixed-width output analysis for Markov chain Monte Carlo., Journal of the American Statistical Association, 101 1537–1547.
  • Khare, K. and Hobert, J. P. (2011). A spectral analytic comparison of trace-class data augmentation algorithms and their sandwich variants., The Annals of Statistics, 39 2585–2606.
  • Lancaster, H. O. (1957). Some properties of the bivariate normal distribution considered in the form of a contingency table., Biometrika, 44 289–292.
  • Liu, J. S. (1994). The collapsed Gibbs sampler in Bayesian computations with applications to a gene regulation problem., Journal of the American Statistical Association, 89 958–966.
  • Liu, J. S., Wong, W. H. and Kong, A. (1994). Covariance structure of the Gibbs sampler with applications to comparisons of estimators and augmentation schemes., Biometrika, 81 27–40.
  • Liu, J. S., Wong, W. H. and Kong, A. (1995). Covariance structure and convergence rate of the Gibbs sampler with various scans., Journal of the Royal Statistical Society, Series B, 57 157–169.
  • Liu, J. S. and Wu, Y. N. (1999). Parameter expansion for data augmentation., Journal of the American Statistical Association, 94 1264–1274.
  • Mira, A. and Geyer, C. J. (1999). Ordering Monte Carlo Markov chains. Tech. rep., School of Statistics, University of, Minnesota.
  • Parthasarathy, K. R. (1967)., Probability Measures on Metric Spaces. Academic Press, New York.
  • Ramachandran, D. (1979)., Perfect Measures: Basic theory. ISI lecture notes, Macmillian.
  • Roberts, G. O. and Rosenthal, J. S. (1997). Geometric ergodicity and hybrid Markov chains., Electronic Communications in Probability, 2 13–25.
  • Tanner, M. A. and Wong, W. H. (1987). The calculation of posterior distributions by data augmentation (with discussion)., Journal of the American Statistical Association, 82 528–550.
  • Yu, K. and Moyeed, R. A. (2001). Bayesian quantile regression., Statistics & Probability Letters, 54 437–447.
  • Yu, Y. and Meng, X. L. (2011). To center or not to center: That is not the question – An Ancillarity-Sufficiency Interweaving Strategy (ASIS) for boosting MCMC efficiency (with discussion)., Journal of Computational and Graphical Statistics, 20 531–615.