Annals of Statistics

Convergence rates for posterior distributions and adaptive estimation

Tzee-Ming Huang

Full-text: Open access


The goal of this paper is to provide theorems on convergence rates of posterior distributions that can be applied to obtain good convergence rates in the context of density estimation as well as regression. We show how to choose priors so that the posterior distributions converge at the optimal rate without prior knowledge of the degree of smoothness of the density function or the regression function to be estimated.

Article information

Ann. Statist., Volume 32, Number 4 (2004), 1556-1593.

First available in Project Euclid: 4 August 2004

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62A15
Secondary: 62G20: Asymptotic properties 62G07: Density estimation

Convergence rate nonparametric regression density estimation Bayesian adaptive estimation sieves


Huang, Tzee-Ming. Convergence rates for posterior distributions and adaptive estimation. Ann. Statist. 32 (2004), no. 4, 1556--1593. doi:10.1214/009053604000000490.

Export citation


  • Barron, A., Birgé, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301–413.
  • Barron, A., Schervish, M. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536–561.
  • Barron, A. and Sheu, C. (1991). Approximation of density functions by sequences of exponentional families. Ann. Statist. 19 1347–1369.
  • Belitser, E. and Ghosal, S. (2003). Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution. Ann. Statist. 31 536–559.
  • Chernoff, H. (1952). A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Ann. Math. Statist. 23 493–507.
  • Diaconis, P. and Freedman, D. (1986). On the consistency of Bayes estimates (with discussion). Ann. Statist. 14 1–67.
  • Doob, J. L. (1949). Application of the theory of martingales. Coll. Int. du CNRS Paris no. 13 23–27.
  • Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999). Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 143–158.
  • Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531.
  • Kolmogorov, A. N. and Tikhomirov, V. M. (1961). $\ep$-entropy and $\ep$-capacity of sets in functional space. Amer. Math. Soc. Transl. Ser. 2 17 277–364.
  • Schumaker, L. L. (1981). Spline Functions: Basic Theory. Wiley, New York.
  • Schwartz, L. (1965). On Bayes procedures. Z. Wahrsch. Verw. Gebiete 4 10–26.
  • Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687–714.
  • Stone, C. (1982). Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10 1040–1053.
  • Stone, C. (1990). Large-sample inference for log-spline models. Ann. Statist. 18 717–741.
  • Yang, Y. and Barron, A. (1998). An asymptotic property of model selection criteria. IEEE Trans. Inform. Theory 44 95–116.