The Annals of Statistics

Rates of convergence of posterior distributions

Xiaotong Shen and Larry Wasserman

Full-text: Open access

Abstract

We compute the rate at which the posterior distribution concentrates around the true parameter value. The spaces we work in are quite general and include in finite dimensional cases. The rates are driven by two quantities: the size of the space, as measured by bracketing entropy, and the degree to which the prior concentrates in a small ball around the true parameter. We consider two examples.

Article information

Source
Ann. Statist., Volume 29, Issue 3 (2001), 687-714.

Dates
First available in Project Euclid: 24 December 2001

Permanent link to this document
https://projecteuclid.org/euclid.aos/1009210686

Digital Object Identifier
doi:10.1214/aos/1009210686

Mathematical Reviews number (MathSciNet)
MR1865337

Zentralblatt MATH identifier
1041.62022

Subjects
Primary: 62A15;secondary .
Secondary: 62E20: Asymptotic distribution theory 62G15: Tolerance and confidence regions

Keywords
Bayesian inference asymptotic inference non-parametric models sieves

Citation

Shen, Xiaotong; Wasserman, Larry. Rates of convergence of posterior distributions. Ann. Statist. 29 (2001), no. 3, 687--714. doi:10.1214/aos/1009210686. https://projecteuclid.org/euclid.aos/1009210686


Export citation

References

  • Barron, A. (1988). The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions. Technical Report 7, Dept. Statistics, Univ. Illinois, Champaign.
  • Barron, A., Schervish, M. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536-561.
  • Barron, A. and Yang, Y. (1998). Anasymptotic property of model selectioncriteria. IEEE Trans. Inform. Theory 44 95-116.
  • Barron, A., Birg´e, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301-413.
  • Berger, J. O. (1986). Discussion on the consistency of Bayes estimates (with discussion). Ann. Statist. 14 1-67.
  • Birman, M. S. and Solomjak, M. Z. (1967). Piecewise-polynomial approximation of functions of the classes Wp. Mat. Sbornik 73 295-317.
  • Brown, L. and Low, M. (1996). Asymptotic equivalence of non-parametric regression and white noise model. Ann. Statist. 24 2384-2398.
  • Cox, D. D. (1993). An analysis of Bayesian inference for non-parametric regression. Ann. Statist. 21 903-924.
  • Dembo, D., Mayer-Wolf, E. and Zeitouni, O. (1995). Exact behavior of Gaussianseminorms. Statist. Probab. Lett. 23 275-280.
  • Dey, D., M ¨uller, P. and Sinha, D. (1998). Practical Nonparametric and Semiparametric Bayesian Statistics. Springer, New York.
  • Diaconis, P. and Freedman, D. (1986). On the consistency of Bayes estimates (with discussion). Ann. Statist. 14 1-67.
  • Diaconis, P. and Freedman, D. (1993). Non-parametric binary regression: A Bayesian approach. Ann. Statist. 21 2108-2137. Diaconis, P. and Freedman, D. (1997a). On the Bernstein-von Mises Theorem with infinite dimensional parameters. Unpublished manuscript. Diaconis, P. and Freedman, D. (1997b). Consistency of Bayes estimates for nonparametric regression: A review. In Festschrift for Lucien Le Cam (D. Pollard, E. Torgersenand G. Yang, eds.) Springer, New York.
  • Diaconis, P. and Freedman, D. (1998). Consistency of Bayes estimates for nonparametric regression: normal theory. Bernoulli 4 411-444.
  • Doob, J. L. (1948). Applicationof the theory of martingales. Coll. Int. du C. N. R. S. Paris 22-28.
  • Freedman, D. (1963). Onthe asymptotic behavior of Bayes estimates inthe discrete case I. Ann. Math. Statist. 34 1386-1403.
  • Freedman, D. (1999). On the Bernstein-Von Mises theorem with infinite dimensional parameters. Ann. Statist. 27.
  • Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1997). Non-informative priors via sieves and packing numbers. In Advances in Statistical Decision Theory and Applications (S. Panchapakesan and N. Balakrishnan, eds.) 119-132. Birkh¨auser, Boston. Ghosal, S., Ghosh, J. K. and Ramamoorthi, R.V. (1999a). Consistency issues in Bayesian nonparametrics. In Asymptotics, Nonparametrics and Time Series: A Tribute to Madan Lal Puri (S. Ghosh, ed.) 639-667. Dekker, New York Ghosal, S., Ghosh, J. K. and Ramamoorthi, R.V. (1999b). Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 143-158.
  • Ghosal, S., Ghosh, J. K. and Van Der Vaart, A. (1998). Convergence rates of posterior distributions. Technical report, Free Univ. Amsterdam. Grenander U. (1981) Abstract Inference. Wiley, New York.
  • Kuo, H. H. (1975). Gaussian Measures on Banach Spaces. Lecture Notes in Math. 463. Springer, Berlin.
  • Shen, X. (1995). Onthe properties of Bayes procedures ingeneral parameter spaces. Technical report, Ohio State Univ.
  • Shen, X. and Wong, W. H. (1994). Convergence rate of sieve estimates. Ann. Statist. 22 580-615.
  • Stone, C. (1982). Optimal global rates of convergence for non-parametric regression. Ann. Statist. 10 1040-1053.
  • Schwartz, L. (1965). OnBayes' procedures. Z. Wahrsch. Verw. Gebiete 4 10-26.
  • Triebel, H. (1983). Theory of Function Spaces. Birkh¨auser, Boston.
  • van de Geer, S. (2000). Empirical Processes in M-Estimation. Cambridge Univ. Press.
  • Wasserman, L. (1998). Asymptotic properties of nonparametric Bayesian procedures. In Practical Nonparametric and Semiparametric Bayesian Statistics (D. Dey, P. M ¨uller and D. Sinha, eds.) 293-304. Springer, New York.
  • Wong, W. H. and Shen, X. (1995). A probability inequality for the likelihood surface and convergence rate of the maximum likelihood estimate. Ann. Statist. 23 339-362.
  • Zeidler, E. (1990). Nonlinear Functional Analysis and its Applications II/A. Springer, New York.
  • Zhao, L. (1993). Frequentist and Bayesian aspects of some nonparametric estimation. Ph.D. dissertation, Cornell Univ.
  • Zhao, L. (1998). A hierarchical Bayesian approach in nonparametric function estimation. Technical report, Dept. Statistics, Univ. Pennsylvania.