The Annals of Statistics

The consistency of posterior distributions in nonparametric problems

Andrew Barron, Mark J. Schervish, and Larry Wasserman

Full-text: Open access

Abstract

We give conditions that guarantee that the posterior probability of every Hellinger neighborhood of the true distribution tends to 1 almost surely. The conditions are (1) a requirement that the prior not put high mass near distributions with very rough densities and (2) a requirement that the prior put positive mass in Kullback-Leibler neighborhoods of the true distribution. The results are based on the idea of approximating the set of distributions with a finite-dimensional set of distributions with sufficiently small Hellinger bracketing metric entropy. We apply the results to some examples.

Article information

Source
Ann. Statist., Volume 27, Number 2 (1999), 536-561.

Dates
First available in Project Euclid: 5 April 2002

Permanent link to this document
https://projecteuclid.org/euclid.aos/1018031206

Digital Object Identifier
doi:10.1214/aos/1018031206

Mathematical Reviews number (MathSciNet)
MR1714718

Zentralblatt MATH identifier
0980.62039

Subjects
Primary: 62G20: Asymptotic properties

Keywords
Exponential families Hellinger distance nonparametric Bayesian inference Pólya trees.

Citation

Barron, Andrew; Schervish, Mark J.; Wasserman, Larry. The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 (1999), no. 2, 536--561. doi:10.1214/aos/1018031206. https://projecteuclid.org/euclid.aos/1018031206


Export citation

References

  • ALEXANDER, K. 1984. Probability inequalities for empirical processes and a law of the iterated logarithm. Ann. Probab. 12 1041 1067. Z.
  • BARRON, A. 1998. Information-theoretic characterization of Bayes performance and the choice Z of priors in parametric and nonparametric problems. In Bayesian Statistics 6 J.. Bernardo, J. Berger, A. Dawid and A. Smith, eds. 27 52. Oxford, New York. Z.
  • BARRON, A. R. 1988. The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions. Technical Report 7, Dept. Statistics, Univ. Illinois, Champaign. Z. Z
  • DIACONIS, P. and FREEDMAN, D. A. 1986. On the consistency of Bayes estimates with discus. sion. Ann. Statist. 14 1 26. Z.
  • DIACONIS, P. and FREEDMAN, D. A. 1997. Consistency of Bayes estimates for nonparametric Z regression: a review. In Festschrift for Lucien Le Cam D. Pollard, E. Torgersen and G.. Yang, eds. 157 165. Springer, New York.
  • DIACONIS, P. and FREEDMAN, D. A. 1998. Consistency of Bayes estimates for nonparametric regression: normal theory. Bernoulli 4 411 444. Z.
  • DOOB, J. L. 1949. Application of the theory of martingales. In Le Calcul des Probabilites et ses ´ Applications 23 27. Colloques Internationaux du Centre National de la Recherche Scientifique, Paris. Z.
  • FABIUS, J. 1964. Asymptotic behavior of Bayes estimates. Ann. Math. Statist. 35 846 856. Z.
  • FREEDMAN, D. A. 1963. On the asymptotic behavior of Bayes' estimates in the discrete case. Ann. Math. Statist. 34 1194 1216. Z.
  • GEMAN, S. and HWANG, C. 1982. Nonparametric maximum likelihood estimation by the method of sieves. Ann. Statist. 10 401 414. Z.
  • GHOSHAL, S., GHOSH, J. K. and RAMAMOORTHI, R. V. 1999A. Consistency issues in Bayesian Z. nonparametrics. In Asymptotics, Nonparametrics and Time series S. Ghosh, ed. 639 667. Dekker, New York. Z.
  • GHOSAL, S., GHOSH, J. K. and RAMAMOORTHI, R. V. 1999B. Consistent semiparametric Bayesian inference about a location parameter. J. Statist. Plann. Inference. To appear. Z.
  • GHOSHAL, S., GHOSH, J. and VAN DER VAART, A. 1998. Convergence rates of posterior distribution. Technical Report 504, Dept. Mathematics, Free University, Amsterdam. Z.
  • GRENANDER, U. 1981. Abstract Inference. Wiley, New York. Z.
  • KRAFT, C. H. 1964. A class of distribution function processes which have derivatives. J. Appl. Probab. 1 385 388. Z.
  • LAVINE, M. 1992. Some aspects of Polya tree distributions for statistical modelling. Ann. ´ Statist. 20 1222 1235. Z.
  • LAVINE, M. 1994. More aspects of Polya tree distributions for statistical modelling. Ann. ´ Statist. 22 1161 1176. Z.
  • LENK, P. J. 1988. The logistic normal distribution for Bayesian, nonparametric, predictive densities. J. Amer. Statist. Assoc. 83 509 516. Z.
  • LENK, P. J. 1991. Towards a practicable Bayesian nonparametric density estimator. Biometrika 78 531 543. Z.
  • LEONARD, T. 1978. Density estimation, stochastic processes, and prior information. J. Roy. Statist. Soc. Ser. B 40 113 146. Z.
  • MAULDIN, R. D., SUDDERTH, W. D. and WILLIAMS, S. C. 1992. Polya trees and random distribu´ tions. Ann. Statist. 20 1203 1221. Z.
  • POLLARD, D. 1991. Bracketing methods in statistics and econometrics. In Nonparametric and Semiparametric Methods in Econometrics and Statistics: Proceedings of the Fifth Z International Symposium in Economic Theory and Econometrics W. A. Barnett, J.. Powell and G. E. Tauchen, eds. 337 355. Cambridge Univ. Press. Z.
  • SCHERVISH, M. J. 1995. Theory of Statistics. Springer, New York. Z.
  • SCHWARTZ, L. 1965. On Bayes procedures. Z. Wahrsch. Verw. Gabiete 4 10 26. Z.
  • SHEN, X. and WASSERMAN, L. 1998. Rates of convergence of posterior distribution. Technical Report 678, Dept. Statistics, Carnegie Mellon Univ. Z.
  • VAN DE GEER, S. 1993. Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Statist. 21 14 44. Z.
  • WONG, W. H. and SHEN, X. 1995. Probability inequalities for likelihood ratios and convergence rates of sieve MLE's. Ann. Statist. 23 339 362.
  • NEW HAVEN, CONNECTICUT 06520 CARNEGIE MELLON UNIVERSITY E-MAIL: barron@stat.yale.edu PITTSBURGH, PENNSYLVANIA 15213 E-MAIL: mark@stat.cmu.edu, larry@stat.cmu.edu