Institute of Mathematical Statistics Collections

Remarks on consistency of posterior distributions

Taeryon Choi and R. V. Ramamoorthi

Full-text: Open access

Abstract

In recent years, the literature in the area of Bayesian asymptotics has been rapidly growing. It is increasingly important to understand the concept of posterior consistency and validate specific Bayesian methods, in terms of consistency of posterior distributions. In this paper, we build up some conceptual issues in consistency of posterior distributions, and discuss panoramic views of them by comparing various approaches to posterior consistency that have been investigated in the literature. In addition, we provide interesting results on posterior consistency that deal with non-exponential consistency, improper priors and non i.i.d. (independent but not identically distributed) observations. We describe a few examples for illustrative purposes.

Chapter information

Source
Bertrand Clarke and Subhashis Ghosal, eds., Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2008), 170-186

Dates
First available in Project Euclid: 28 April 2008

Permanent link to this document
https://projecteuclid.org/euclid.imsc/1209398468

Digital Object Identifier
doi:10.1214/074921708000000138

Mathematical Reviews number (MathSciNet)
MR2459224

Subjects
Primary: 62G20: Asymptotic properties
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures 62F15: Bayesian inference

Keywords
affinity Doob’s theorem exponential consistency posterior distribution strongly separated

Rights
Copyright © 2008, Institute of Mathematical Statistics

Citation

Choi, Taeryon; Ramamoorthi, R. V. Remarks on consistency of posterior distributions. Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, 170--186, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2008. doi:10.1214/074921708000000138. https://projecteuclid.org/euclid.imsc/1209398468


Export citation

References

  • [1] Amewou-Atisso, M., Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (2003). Posterior consistency for semi-parametric regression problems. Bernoulli 9 291–312.
  • [2] Barron, A. (1988). The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions. Technical Report 7, Dept. Statistics, Univ. Illinois, Champaign.
  • [3] Barron, A., Schervish, M. J. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536–561.
  • [4] Birgé, L. (1983). Robust testing for independent non-identically distributed variables and Markov chains. In Practical Nonparametric and Semiparametric Bayesian Statistics. Lecture Notes in Statist. 16 (D. Dey, P. Müller and D. Sinha, eds.) 134–162. Springer, New York.
  • [5] Chatterji, S. D. and Mandrekar, V. (1978). Equivalence and Singularity of Gaussian Measures and Applications. Probabilistic Analysis and Related Topics 1. Academic Press, New York.
  • [6] Choi, T. (2005). Posterior consistency in nonparametric regression problems under Gaussian process priors. Ph.D. thesis, Carnegie Mellon Univ., Pittsburgh, PA.
  • [7] Choi, T. (2007). Alternative posterior consistency results in nonparametric binary regression using gaussian process priors. J. Statist. Plann. Inference 137 2975–2983.
  • [8] Choi, T. and Schervish, M. J. (2007). On posterior consistency in nonparametric regression problems. J. Multivariate Anal. 98 1969–1987.
  • [9] Choudhuri, N., Ghosal, S. and Roy, A. (2004). Bayesian estimation of the spectral density of a time series. J. Amer. Statist. Assoc. 99 1050–1059.
  • [10] Diaconis, P. and Freedman, D. A. (1986). On the consistency of Bayes estimates. Ann. Statist. 14 1–26.
  • [11] Doob, J. L. (1949). Application of the theory of martingales. Coll. Int. du C. N. R. S. Paris. 23–27.
  • [12] Freedman, D. A. (1963). On the asymptotic behavior of Bayes’ estimates in the discrete case. Ann. Math. Statist. 34 1386–1403.
  • [13] Freedman, D. A. (1965). On the asymptotic behavior of Bayes’ estimates in the discrete case. II. Ann. Math. Statist. 36 454–456.
  • [14] Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999a). Consistency issues in Bayesian nonparametrics. In Asymptotics, Nonparametrics, and Time Series. Statist. Textbooks Monogr. 158 639–667. Dekker, New York.
  • [15] Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999b). Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 143–158.
  • [16] Ghosal, S. and Roy, A. (2006). Posterior consistency of Gaussian process prior for nonparametric binary regression. Ann. Statist. 34 2413–2429.
  • [17] Ghosal, S. and van der Vaart, A. W. (2007). Convergence rates of posterior distributions for noniid observations. Ann. Statist. 35 192–223.
  • [18] Ghosh, J. K., Delampady, M. and Samanta, T. (2006). An Introduction to Bayesian Analysis Theory and Methods. Springer, New York.
  • [19] Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics. Springer, New York.
  • [20] Kass, R. E. and Wasserman, L. (1996). The selection of prior distributions by formal rules. J. Amer. Statist. Assoc. 91 1343–1370.
  • [21] Kraft, C. (1955). Some conditions for consistency and uniform consistency of statistical procedures. Univ. California Publ. Statist. 2 125–141.
  • [22] Le Cam, L. M. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York.
  • [23] Li, W. and Shao, Q.-M. (2001). Gaussian processes: inequalities, small ball probabilities and applications. In Stochastic Processes: Theory and Methods. Handbook of Statist. 19 533–597. North-Holland, Amsterdam.
  • [24] Salinetti, G. (2003). New tools for consistency in Bayesian nonparametrics. In Bayesian Statistics 7 369–384. Oxford Univ. Press, New York.
  • [25] Schwartz, L. (1965). On Bayes procedures. Z. Wahrsch. Verw. Gebiete 4 10–26.
  • [26] Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 666–686.
  • [27] Shepp, L. A. (1965). Distingunishing a sequence of random variables from a translate of itself. Ann. Math. Statist. 36 1107–1112.
  • [28] Srivastava, S. M. (1998). A Course on Borel Sets. Springer, Berlin.
  • [29] Tokdar, S. and Ghosh, J. K. (2007). Posterior consistency of Gaussian process priors in density estimation. J. Statist. Plann. Inference 137 34–42.
  • [30] van de Geer, S. (1993). Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Statist. 21 14–44.
  • [31] van der Vaart, A. W. and van Zanten, J. (2007). Rates of contraction of posterior distributions based on Gaussian process priors. Ann. Statist. To appear.
  • [32] Walker, S. (2004). New approaches to Bayesian consistency. Ann. Statist. 32 2028–2043.
  • [33] Walker, S. G. (2003). Bayesian consistency for a class of regression problems. South African Stat. J. 37 149–167.