The Annals of Statistics

Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator

Judith Rousseau and Botond Szabo

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

We consider the asymptotic behaviour of the marginal maximum likelihood empirical Bayes posterior distribution in general setting. First, we characterize the set where the maximum marginal likelihood estimator is located with high probability. Then we provide oracle type of upper and lower bounds for the contraction rates of the empirical Bayes posterior. We also show that the hierarchical Bayes posterior achieves the same contraction rate as the maximum marginal likelihood empirical Bayes posterior. We demonstrate the applicability of our general results for various models and prior distributions by deriving upper and lower bounds for the contraction rates of the corresponding empirical and hierarchical Bayes posterior distributions.

Article information

Source
Ann. Statist. Volume 45, Number 2 (2017), 833-865.

Dates
Received: April 2015
Revised: March 2016
First available in Project Euclid: 16 May 2017

Permanent link to this document
https://projecteuclid.org/euclid.aos/1494921959

Digital Object Identifier
doi:10.1214/16-AOS1469

Zentralblatt MATH identifier
1371.62048

Subjects
Primary: 62G20: Asymptotic properties 62G05: Estimation 60K35: Interacting random processes; statistical mechanics type models; percolation theory [See also 82B43, 82C43]
Secondary: 62G08: Nonparametric regression 62G07: Density estimation

Keywords
Posterior contraction rates adaptation empirical Bayes hierarchical Bayes nonparametric regression density estimation Gaussian prior truncation prior

Citation

Rousseau, Judith; Szabo, Botond. Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator. Ann. Statist. 45 (2017), no. 2, 833--865. doi:10.1214/16-AOS1469. https://projecteuclid.org/euclid.aos/1494921959


Export citation

References

  • [1] Arbel, J., Gayraud, G. and Rousseau, J. (2013). Bayesian optimal adaptive estimation using a sieve prior. Scand. J. Stat. 40 549–570.
  • [2] Babenko, A. and Belitser, E. (2010). Oracle convergence rate of posterior under projection prior and Bayesian model selection. Math. Methods Statist. 19 219–245.
  • [3] Belitser, E. and Enikeeva, F. (2008). Empirical Bayesian test of the smoothness. Math. Methods Statist. 17 1–18.
  • [4] Belitser, E. and Ghosal, S. (2003). Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution. Ann. Statist. 31 536–559.
  • [5] Castillo, I. (2008). Lower bounds for posterior rates with Gaussian process priors. Electron. J. Stat. 2 1281–1299.
  • [6] Castillo, I. and Rousseau, J. (2015). A Bernstein–von Mises theorem for smooth functionals in semiparametric models. Ann. Statist. 43 2353–2383.
  • [7] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903–923.
  • [8] Donnet, S., Rivoirard, V., Rousseau, J. and Scricciolo, C. (2014). Posterior concentration rates for empirical Bayes procedures, with applications to Dirichlet Process mixtures. Available at arXiv:1406.4406v1.
  • [9] Florens, J.-P. and Simoni, A. (2012). Regularized posteriors in linear ill-posed inverse problems. Scand. J. Stat. 39 214–235.
  • [10] Gao, C. and Zhou, H. H. (2016). Rate exact Bayesian adaptation with modified block priors. Ann. Statist. 44 318–345.
  • [11] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531.
  • [12] Ghosal, S. and van der Vaart, A. (2007). Convergence rates of posterior distributions for non-i.i.d. observations. Ann. Statist. 35 192–223.
  • [13] Johnstone, I. M. and Silverman, B. W. (2004). Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences. Ann. Statist. 32 1594–1649.
  • [14] Knapik, B. T., Szabó, B. T., van der Vaart, A. W. and van Zanten, J. H. (2016). Bayes procedures for adaptive inference in inverse problems for the white noise model. Probab. Theory Related Fields 164 771–813.
  • [15] Knapik, B. T., van der Vaart, A. W. and van Zanten, J. H. (2011). Bayesian inverse problems with Gaussian priors. Ann. Statist. 39 2626–2657.
  • [16] Kuelbs, J. and Li, W. V. (1993). Metric entropy and the small ball problem for Gaussian measures. J. Funct. Anal. 116 133–157.
  • [17] Lehmann, E. L. and Casella, G. (1998). Theory of Point Estimation, 2nd ed. Springer, New York.
  • [18] Lian, H. (2014). Adaptive rates of contraction of posterior distributions in Bayesian wavelet regression. J. Statist. Plann. Inference 145 92–101.
  • [19] Petrone, S., Rousseau, J. and Scricciolo, C. (2014). Bayes and empirical Bayes: Do they merge? Biometrika 101 285–302.
  • [20] Ray, K. (2013). Bayesian inverse problems with non-conjugate priors. Electron. J. Stat. 7 2516–2549.
  • [21] Rivoirard, V. and Rousseau, J. (2012). Bernstein–von Mises theorem for linear functionals of the density. Ann. Statist. 40 1489–1523.
  • [22] Rivoirard, V. and Rousseau, J. (2012). Posterior concentration rates for infinite dimensional exponential families. Bayesian Anal. 7 311–333.
  • [23] Rousseau, J. and Szabo, B. (2016). Supplement to “Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator.” DOI:10.1214/16-AOS1469SUPP.
  • [24] Serra, P. and Krivobokova, T. (2014). Adaptive empirical Bayesian smoothing splines. Available at arXiv:1411.6860.
  • [25] Shen, W. and Ghosal, S. (2015). Adaptive Bayesian procedures using random series priors. Scand. J. Stat. 42 1194–1213.
  • [26] Sniekers, S. and van der Vaart, A. (2015). Adaptive Bayesian credible sets in regression with a Gaussian process prior. Electron. J. Stat. 9 2475–2527.
  • [27] Szabó, B., van der Vaart, A. and van Zanten, H. (2015). Honest Bayesian confidence sets for the $L^{2}$-norm. J. Statist. Plann. Inference 166 36–51.
  • [28] Szabó, B., van der Vaart, A. W. and van Zanten, J. H. (2015). Frequentist coverage of adaptive nonparametric Bayesian credible sets. Ann. Statist. 43 1391–1428.
  • [29] Szabó, B. T., van der Vaart, A. W. and van Zanten, J. H. (2013). Empirical Bayes scaling of Gaussian priors in the white noise model. Electron. J. Stat. 7 991–1018.
  • [30] Tsybakov, A. B. (2004). Introduction à L’estimation Non-paramétrique. Mathématiques & Applications (Berlin) [Mathematics & Applications] 41. Springer, Berlin.
  • [31] van der Vaart, A. W. and van Zanten, J. H. (2008). Rates of contraction of posterior distributions based on Gaussian process priors. Ann. Statist. 36 1435–1463.
  • [32] van der Vaart, A. W. and van Zanten, J. H. (2008). Reproducing kernel Hilbert spaces of Gaussian priors. In Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh. Inst. Math. Stat. Collect. 3 200–222. IMS, Beachwood, OH.
  • [33] Verdinelli, I. and Wasserman, L. (1998). Bayesian goodness-of-fit testing using infinite-dimensional exponential families. Ann. Statist. 26 1215–1241.
  • [34] Zhao, L. H. (2000). Bayesian aspects of some nonparametric problems. Ann. Statist. 28 532–552.

Supplemental materials

  • Supplement to “Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator”. This is the supplementary material associated to the present paper. We provide here the proofs of Propositions 3.1–3.6, together with some technical Lemmas used in the context of priors (T2) and (T3) and some technical Lemmas used in the study of the hierarchical Bayes posteriors. Finally some Lemmas used in the regression and density estimation problems are given.