The Annals of Statistics

Adaptive nonparametric Bayesian inference using location-scale mixture priors

R. de Jonge and J. H. van Zanten

Full-text: Open access

Abstract

We study location-scale mixture priors for nonparametric statistical problems, including multivariate regression, density estimation and classification. We show that a rate-adaptive procedure can be obtained if the prior is properly constructed. In particular, we show that adaptation is achieved if a kernel mixture prior on a regression function is constructed using a Gaussian kernel, an inverse gamma bandwidth, and Gaussian mixing weights.

Article information

Source
Ann. Statist., Volume 38, Number 6 (2010), 3300-3320.

Dates
First available in Project Euclid: 20 September 2010

Permanent link to this document
https://projecteuclid.org/euclid.aos/1284988407

Digital Object Identifier
doi:10.1214/10-AOS811

Mathematical Reviews number (MathSciNet)
MR2766853

Zentralblatt MATH identifier
1204.62062

Subjects
Primary: 62G08: Nonparametric regression 62C10: Bayesian problems; characterization of Bayes procedures
Secondary: 62G20: Asymptotic properties

Keywords
Rate of convergence posterior distribution adaptation Bayesian inference nonparametric regression kernel mixture priors

Citation

de Jonge, R.; van Zanten, J. H. Adaptive nonparametric Bayesian inference using location-scale mixture priors. Ann. Statist. 38 (2010), no. 6, 3300--3320. doi:10.1214/10-AOS811. https://projecteuclid.org/euclid.aos/1284988407


Export citation

References

  • [1] Banerjee, S., Gelfand, A. E., Finley, A. O. and Sang, H. (2008). Gaussian predictive process models for large spatial data sets. J. R. Stat. Soc. Ser. B Stat. Methodol. 70 825–848.
  • [2] Belitser, E. and Ghosal, S. (2003). Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution. Ann. Statist. 31 536–559.
  • [3] Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999). Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 143–158.
  • [4] Ghosal, S. (2001). Convergence rates for density estimation with Bernstein polynomials. Ann. Statist. 29 1264–1280.
  • [5] Ghosal, S. and van der Vaart, A. W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist. 29 1233–1263.
  • [6] Ghosal, S. and van der Vaart, A. (2007). Posterior convergence rates of Dirichlet mixtures at smooth densities. Ann. Statist. 35 697–723.
  • [7] Ghosal, S. and van der Vaart, A. W. (2007). Convergence rates for posterior distributions for noniid observations. Ann. Statist. 35 192–223.
  • [8] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531.
  • [9] Ghosal, S., Lember, J. and Van Der Vaart, A. (2003). On Bayesian adaptation. In Proceedings of the Eighth Vilnius Conference on Probability Theory and Mathematical Statistics, Part II (2002). Acta Appl. Math. 79 165–175.
  • [10] Ghosal, S., Lember, J. and Van Der Vaart, A. (2008). Nonparametric Bayesian model selection and averaging. Electron. J. Stat. 2 63–89.
  • [11] Higdon, D. (2002). Space and space-time modeling using process convolutions. In Quantitative Methods for Current Environmental Issues 37–56. Springer, London.
  • [12] Huang, T.-M. (2004). Convergence rates for posterior distributions and adaptive estimation. Ann. Statist. 32 1556–1593.
  • [13] Kolmogorov, A. N. and Tihomirov, V. M. (1961). ɛ-entropy and ɛ-capacity of sets in functional space. Amer. Math. Soc. Transl. Ser. 2 17 277–364.
  • [14] Kruijer, W. and van der Vaart, A. (2008). Posterior convergence rates for Dirichlet mixtures of beta densities. J. Statist. Plann. Inference 138 1981–1992.
  • [15] Kruijer, W., Rousseau, J. and van der Vaart, A. W. (2009). Adaptive Bayesian density estimation with location-scale mixtures. Preprint, Univ. Paris Dauphine.
  • [16] Kuelbs, J. and Li, W. V. (1993). Metric entropy and the small ball problem for Gaussian measures. J. Funct. Anal. 116 133–157.
  • [17] Lember, J. and van der Vaart, A. W. (2007). On universal Bayesian adaptation. Statist. Decisions 25 127–152.
  • [18] Li, W. V. and Linde, W. (1999). Approximation, metric entropy and small ball estimates for Gaussian measures. Ann. Probab. 27 1556–1578.
  • [19] Lifshits, M. A. (1995). Gaussian Random Functions. Mathematics and Its Applications 322. Kluwer Academic, Dordrecht.
  • [20] Petrone, S. and Wasserman, L. (2002). Consistency of Bernstein polynomial posteriors. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 79–100.
  • [21] Rousseau, J. (2010). Rates of convergence for the posterior distributions of mixtures of betas and adaptive nonparametric estimation of the density. Ann. Statist. 38 146–180.
  • [22] Short, M. B., Higdon, D. M. and Kronberg, P. P. (2007). Estimation of Faraday rotation measures of the near galactic sky using Gaussian process models. Bayesian Anal. 2 665–680.
  • [23] Tokdar, S. T. (2006). Posterior consistency of Dirichlet location-scale mixture of normals in density estimation and regression. Sankhyā 68 90–110.
  • [24] van der Meulen, F. H., van der Vaart, A. W. and van Zanten, J. H. (2006). Convergence rates of posterior distributions for Brownian semimartingale models. Bernoulli 12 863–888.
  • [25] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York.
  • [26] van der Vaart, A. W. and van Zanten, J. H. (2009). Adapative Bayesian estimation using a Gaussian random field with inverse gamma bandwidth. Ann. Statist. 37 2655–2675.
  • [27] Van der Vaart, A. W. and Van Zanten, J. H. (2008). Rates of contraction of posterior distributions based on Gaussian process priors. Ann. Statist. 36 1435–1463.
  • [28] Van der Vaart, A. W. and Van Zanten, J. H. (2008). Reproducing kernel Hilbert spaces of Gaussian priors. In Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh (B. Clarke and S. Ghosal, eds.) 200–222. IMS, Beachwood, OH.
  • [29] Wu, Y. and Ghosal, S. (2008). Kullback Leibler property of kernel mixture priors in Bayesian density estimation. Electron. J. Stat. 2 298–331.