The Annals of Statistics

Convergence rates for posterior distributions and adaptive estimation

Tzee-Ming Huang

Full-text: Open access

Abstract

The goal of this paper is to provide theorems on convergence rates of posterior distributions that can be applied to obtain good convergence rates in the context of density estimation as well as regression. We show how to choose priors so that the posterior distributions converge at the optimal rate without prior knowledge of the degree of smoothness of the density function or the regression function to be estimated.

Article information

Source
Ann. Statist., Volume 32, Number 4 (2004), 1556-1593.

Dates
First available in Project Euclid: 4 August 2004

Permanent link to this document
https://projecteuclid.org/euclid.aos/1091626179

Digital Object Identifier
doi:10.1214/009053604000000490

Mathematical Reviews number (MathSciNet)
MR2089134

Zentralblatt MATH identifier
1095.62055

Subjects
Primary: 62A15
Secondary: 62G20: Asymptotic properties 62G07: Density estimation

Keywords
Convergence rate nonparametric regression density estimation Bayesian adaptive estimation sieves

Citation

Huang, Tzee-Ming. Convergence rates for posterior distributions and adaptive estimation. Ann. Statist. 32 (2004), no. 4, 1556--1593. doi:10.1214/009053604000000490. https://projecteuclid.org/euclid.aos/1091626179


Export citation

References

  • Barron, A., Birgé, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301--413.
  • Barron, A., Schervish, M. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536--561.
  • Barron, A. and Sheu, C. (1991). Approximation of density functions by sequences of exponentional families. Ann. Statist. 19 1347--1369.
  • Belitser, E. and Ghosal, S. (2003). Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution. Ann. Statist. 31 536--559.
  • Chernoff, H. (1952). A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Ann. Math. Statist. 23 493--507.
  • Diaconis, P. and Freedman, D. (1986). On the consistency of Bayes estimates (with discussion). Ann. Statist. 14 1--67.
  • Doob, J. L. (1949). Application of the theory of martingales. Coll. Int. du CNRS Paris no. 13 23--27.
  • Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999). Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 143--158.
  • Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500--531.
  • Kolmogorov, A. N. and Tikhomirov, V. M. (1961). $\ep$-entropy and $\ep$-capacity of sets in functional space. Amer. Math. Soc. Transl. Ser. 2 17 277--364.
  • Schumaker, L. L. (1981). Spline Functions: Basic Theory. Wiley, New York.
  • Schwartz, L. (1965). On Bayes procedures. Z. Wahrsch. Verw. Gebiete 4 10--26.
  • Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687--714.
  • Stone, C. (1982). Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10 1040--1053.
  • Stone, C. (1990). Large-sample inference for log-spline models. Ann. Statist. 18 717--741.
  • Yang, Y. and Barron, A. (1998). An asymptotic property of model selection criteria. IEEE Trans. Inform. Theory 44 95--116.