The Annals of Statistics

On adaptive posterior concentration rates

Marc Hoffmann, Judith Rousseau, and Johannes Schmidt-Hieber

Full-text: Open access

Abstract

We investigate the problem of deriving posterior concentration rates under different loss functions in nonparametric Bayes. We first provide a lower bound on posterior coverages of shrinking neighbourhoods that relates the metric or loss under which the shrinking neighbourhood is considered, and an intrinsic pre-metric linked to frequentist separation rates. In the Gaussian white noise model, we construct feasible priors based on a spike and slab procedure reminiscent of wavelet thresholding that achieve adaptive rates of contraction under $L^{2}$ or $L^{\infty}$ metrics when the underlying parameter belongs to a collection of Hölder balls and that moreover achieve our lower bound. We analyse the consequences in terms of asymptotic behaviour of posterior credible balls as well as frequentist minimax adaptive estimation. Our results are appended with an upper bound for the contraction rate under an arbitrary loss in a generic regular experiment. The upper bound is attained for certain sieve priors and enables to extend our results to density estimation.

Article information

Source
Ann. Statist., Volume 43, Number 5 (2015), 2259-2295.

Dates
Received: May 2013
Revised: April 2015
First available in Project Euclid: 16 September 2015

Permanent link to this document
https://projecteuclid.org/euclid.aos/1442364152

Digital Object Identifier
doi:10.1214/15-AOS1341

Mathematical Reviews number (MathSciNet)
MR3396985

Zentralblatt MATH identifier
1327.62306

Subjects
Primary: 62G20: Asymptotic properties 62G08: Nonparametric regression
Secondary: 62G07: Density estimation

Keywords
Bayesian nonparametrics minimax adaptive estimation posterior concentration rates sup-norm rates of convergence

Citation

Hoffmann, Marc; Rousseau, Judith; Schmidt-Hieber, Johannes. On adaptive posterior concentration rates. Ann. Statist. 43 (2015), no. 5, 2259--2295. doi:10.1214/15-AOS1341. https://projecteuclid.org/euclid.aos/1442364152


Export citation

References

  • [1] Arbel, J., Gayraud, G. and Rousseau, J. (2013). Bayesian optimal adaptive estimation using a sieve prior. Scand. J. Stat. 40 549–570.
  • [2] Barron, A. (1988). The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions. Technical report, Univ. Illinois at Urbana-Campaign.
  • [3] Belitser, E. and Ghosal, S. (2003). Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution. Ann. Statist. 31 536–559.
  • [4] Brown, L. D. and Low, M. G. (1996). A constrained risk inequality with applications to nonparametric functional estimation. Ann. Statist. 24 2524–2535.
  • [5] Cai, T. T. (2008). On information pooling, adaptability and superefficiency in nonparametric function estimation. J. Multivariate Anal. 99 421–436.
  • [6] Cai, T. T. and Low, M. G. (2004). An adaptation theory for nonparametric confidence intervals. Ann. Statist. 32 1805–1840.
  • [7] Cai, T. T. and Low, M. G. (2005). Adaptive estimation of linear functionals under different performance measures. Bernoulli 11 341–358.
  • [8] Cai, T. T. and Low, M. G. (2006). Adaptation under probabilistic error for estimating linear functionals. J. Multivariate Anal. 97 231–245.
  • [9] Cai, T. T., Low, M. G. and Zhao, L. H. (2007). Trade-offs between global and local risks in nonparametric function estimation. Bernoulli 13 1–19.
  • [10] Castillo, I., Kerkyacharian, G. and Picard, D. (2014). Thomas Bayes’ walk on manifolds. Probab. Theory Related Fields 158 665–710.
  • [11] Choudhuri, N., Ghosal, S. and Roy, A. (2004). Bayesian estimation of the spectral density of a time series. J. Amer. Statist. Assoc. 99 1050–1059.
  • [12] Cohen, A. (2003). Numerical Analysis of Wavelet Methods. North-Holland, Amsterdam.
  • [13] Cohen, A., Daubechies, I. and Vial, P. (1993). Wavelets on the interval and fast wavelet transforms. Appl. Comput. Harmon. Anal. 1 54–81.
  • [14] Donoho, D. and Liu, R. G. (1991). Geometrizing rates of convergence III. Ann. Statist. 19 668–701.
  • [15] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531.
  • [16] Ghosal, S. and van der Vaart, A. (2006). Convergence rates of posterior distributions for non-i.i.d. observations. Ann. Statist. 35 192–223.
  • [17] Giné, E. and Nickl, R. (2011). Rates on contraction for posterior distributions in $L^{r}$-metrics, $1\leq r\leq\infty$. Ann. Statist. 39 2883–2911.
  • [18] Hoffmann, M. and Nickl, R. (2011). On adaptive inference and confidence bands. Ann. Statist. 39 2383–2409.
  • [19] Kruijer, W., Rousseau, J. and van der Vaart, A. (2010). Adaptive Bayesian density estimation with location-scale mixtures. Electron. J. Stat. 4 1225–1257.
  • [20] Lepskiĭ, O. V. (1990). A problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl. 35 454–466.
  • [21] Le Cam, L. and Yang, G. L. (2000). Asymptotics in Statistics: Some Basic Concepts, 2nd ed. Springer, New York.
  • [22] Low, M. G. (1997). On nonparametric confidence intervals. Ann. Statist. 25 2547–2554.
  • [23] Robert, C. (2004). The Bayesian Choice. Springer, New York.
  • [24] Rousseau, J. (2010). Rates of convergence for the posterior distributions of mixtures of betas and adaptive nonparametric estimation of the density. Ann. Statist. 38 146–180.
  • [25] Rousseau, J., Chopin, N. and Liseo, B. (2012). Bayesian nonparametric estimation of the spectral density of a long or intermediate memory Gaussian process. Ann. Statist. 40 964–995.
  • [26] Schwartz, L. (1965). On Bayes procedures. Z. Wahrsch. Verw. Gebiete 4 10–26.
  • [27] Scricciolo, C. (2014). Adaptive Bayesian density estimation in $L^{p}$-metrics with Pitman–Yor or normalized inverse-Gaussian process kernel mixtures. Bayesian Anal. 9 475–520.
  • [28] Shen, W., Tokdar, S. and Ghosal, S. (2012). Adaptive Bayesian multivariate density estimation with Dirichlet mixtures. Technical report.
  • [29] Tang, Y. and Ghosal, S. (2007). Posterior consistency of Dirichlet mixtures for estimating a transition density. J. Statist. Plann. Inference 137 1711–1726.
  • [30] van der Vaart, A. W. and van Zanten, J. H. (2009). Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth. Ann. Statist. 37 2655–2675.
  • [31] Zhao, L. H. (2000). Bayesian aspects of some nonparametric problems. Ann. Statist. 28 532–552.