The Annals of Statistics

Rates of contraction for posterior distributions in Lr-metrics, 1 ≤ r ≤ ∞

Evarist Giné and Richard Nickl

Full-text: Open access

Abstract

The frequentist behavior of nonparametric Bayes estimates, more specifically, rates of contraction of the posterior distributions to shrinking Lr-norm neighborhoods, 1 ≤ r ≤ ∞, of the unknown parameter, are studied. A theorem for nonparametric density estimation is proved under general approximation-theoretic assumptions on the prior. The result is applied to a variety of common examples, including Gaussian process, wavelet series, normal mixture and histogram priors. The rates of contraction are minimax-optimal for 1 ≤ r ≤ 2, but deteriorate as r increases beyond 2. In the case of Gaussian nonparametric regression a Gaussian prior is devised for which the posterior contracts at the optimal rate in all Lr-norms, 1 ≤ r ≤ ∞.

Article information

Source
Ann. Statist., Volume 39, Number 6 (2011), 2883-2911.

Dates
First available in Project Euclid: 24 January 2012

Permanent link to this document
https://projecteuclid.org/euclid.aos/1327413772

Digital Object Identifier
doi:10.1214/11-AOS924

Mathematical Reviews number (MathSciNet)
MR3012395

Zentralblatt MATH identifier
1246.62095

Subjects
Primary: 62G20: Asymptotic properties
Secondary: 62G07: Density estimation 62G08: Nonparametric regression

Keywords
Rate of contraction posterior nonparametric hypothesis testing

Citation

Giné, Evarist; Nickl, Richard. Rates of contraction for posterior distributions in L r -metrics, 1 ≤ r ≤ ∞. Ann. Statist. 39 (2011), no. 6, 2883--2911. doi:10.1214/11-AOS924. https://projecteuclid.org/euclid.aos/1327413772


Export citation

References

  • [1] Barron, A., Schervish, M. J. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536–561.
  • [2] Birgé, L. (1983). Approximation dans les espaces métriques et théorie de l’estimation. Z. Wahrsch. Verw. Gebiete 65 181–237.
  • [3] Bogachev, V. I. (1998). Gaussian Measures. Mathematical Surveys and Monographs 62. Amer. Math. Soc., Providence, RI.
  • [4] Borell, C. (1975). The Brunn–Minkowski inequality in Gauss space. Invent. Math. 30 207–216.
  • [5] Bousquet, O. (2003). Concentration inequalities for sub-additive functions using the entropy method. In Stochastic Inequalities and Applications. Progress in Probability 56 213–247. Birkhäuser, Basel.
  • [6] Castillo, I. (2011). A semiparametric Bernstein–von Mises theorem for Gaussian process priors. Probab. Theory Related Fields. To appear.
  • [7] Ciesielski, Z., Kerkyacharian, G. and Roynette, B. (1993). Quelques espaces fonctionnels associés à des processus gaussiens. Studia Math. 107 171–204.
  • [8] Cohen, A., Daubechies, I. and Vial, P. (1993). Wavelets on the interval and fast wavelet transforms. Appl. Comput. Harmon. Anal. 1 54–81.
  • [9] Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999). Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 143–158.
  • [10] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531.
  • [11] Ghosal, S. and van der Vaart, A. (2007). Convergence rates of posterior distributions for non-i.i.d. observations. Ann. Statist. 35 192–223.
  • [12] Ghosal, S. and van der Vaart, A. (2007). Posterior convergence rates of Dirichlet mixtures at smooth densities. Ann. Statist. 35 697–723.
  • [13] Ghosal, S. and van der Vaart, A. W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist. 29 1233–1263.
  • [14] Giné, E. and Guillou, A. (2002). Rates of strong uniform consistency for multivariate kernel density estimators. Ann. Inst. Henri Poincaré Probab. Stat. 38 907–921.
  • [15] Giné, E. and Mason, D. (2007). On local U-statistic processes and the estimation of densities of functions of several sample variables. Ann. Statist. 35 1105–1145.
  • [16] Giné, E. and Nickl, R. (2008). Adaptation on the space of finite signed measures. Math. Methods Statist. 17 113–122.
  • [17] Giné, E. and Nickl, R. (2009). Uniform limit theorems for wavelet density estimators. Ann. Probab. 37 1605–1646.
  • [18] Giné, E. and Nickl, R. (2011). Supplement to “Rates of contraction for posterior distributions in Lr-metrics, 1 ≤ r ≤ ∞.” DOI:10.1214/11-AOS924SUPP.
  • [19] Härdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. (1998). Wavelets, Approximation, and Statistical Applications. Lecture Notes in Statistics 129. Springer, New York.
  • [20] Ingster, Y. I. (1993). Asymptotically minimax hypothesis testing for nonparametric alternatives. I. Math. Methods Statist. 2 85–114.
  • [21] Le Cam, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York.
  • [22] Ledoux, M. and Talagrand, M. (1991). Probability in Banach Spaces: Isoperimetry and Processes. Ergebnisse der Mathematik und Ihrer Grenzgebiete (3) [Results in Mathematics and Related Areas (3)] 23. Springer, Berlin.
  • [23] Lenk, P. J. (1991). Towards a practicable Bayesian nonparametric density estimator. Biometrika 78 531–543.
  • [24] Li, W. V. and Linde, W. (1998). Existence of small ball constants for fractional Brownian motions. C. R. Acad. Sci. Paris Sér. I Math. 326 1329–1334.
  • [25] Lounici, K. and Nickl, R. (2011). Global uniform risk bounds for wavelet deconvolution estimators. Ann. Statist. 39 201–231.
  • [26] Meyer, Y. (1992). Wavelets and Operators. Cambridge Studies in Advanced Mathematics 37. Cambridge Univ. Press, Cambridge.
  • [27] Nickl, R. (2007). Donsker-type theorems for nonparametric maximum likelihood estimators. Probab. Theory Related Fields 138 411–449.
  • [28] Schwartz, L. (1965). On Bayes procedures. Z. Wahrsch. Verw. Gebiete 4 10–26.
  • [29] Scricciolo, C. (2007). On rates of convergence for Bayesian density estimation. Scand. J. Statist. 34 626–642.
  • [30] Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687–714.
  • [31] Talagrand, M. (1996). New concentration inequalities in product spaces. Invent. Math. 126 505–563.
  • [32] van der Vaart, A. W. and van Zanten, J. H. (2008). Rates of contraction of posterior distributions based on Gaussian process priors. Ann. Statist. 36 1435–1463.
  • [33] van der Vaart, A. W. and van Zanten, J. H. (2008). Reproducing kernel Hilbert spaces of Gaussian priors. In Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh. Inst. Math. Stat. Collect. 3 200–222. IMS, Beachwood, OH.
  • [34] van der Vaart, A. W. and van Zanten, J. H. (2009). Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth. Ann. Statist. 37 2655–2675.
  • [35] Zhao, L. H. (2000). Bayesian aspects of some nonparametric problems. Ann. Statist. 28 532–552.

Supplemental materials

  • Supplementary material: Supplement to “Rates of contraction for posterior distributions in Lr-metrics, 1 ≤ r ≤ ∞ ”. This supplement contains a detailed proof of Lemma 1 and an expanded proof of Proposition 2 from the mentioned article.