## Bernoulli

• Bernoulli
• Volume 24, Number 1 (2018), 231-256.

### Posterior concentration rates for empirical Bayes procedures with applications to Dirichlet process mixtures

#### Abstract

We provide conditions on the statistical model and the prior probability law to derive contraction rates of posterior distributions corresponding to data-dependent priors in an empirical Bayes approach for selecting prior hyper-parameter values. We aim at giving conditions in the same spirit as those in the seminal article of Ghosal and van der Vaart [Ann. Statist. 35 (2007) 192–223]. We then apply the result to specific statistical settings: density estimation using Dirichlet process mixtures of Gaussian densities with base measure depending on data-driven chosen hyper-parameter values and intensity function estimation of counting processes obeying the Aalen model. In the former setting, we also derive recovery rates for the related inverse problem of density deconvolution. In the latter, a simulation study for inhomogeneous Poisson processes illustrates the results.

#### Article information

Source
Bernoulli, Volume 24, Number 1 (2018), 231-256.

Dates
Revised: May 2016
First available in Project Euclid: 27 July 2017

https://projecteuclid.org/euclid.bj/1501142441

Digital Object Identifier
doi:10.3150/16-BEJ872

Mathematical Reviews number (MathSciNet)
MR3706755

Zentralblatt MATH identifier
06778326

#### Citation

Donnet, Sophie; Rivoirard, Vincent; Rousseau, Judith; Scricciolo, Catia. Posterior concentration rates for empirical Bayes procedures with applications to Dirichlet process mixtures. Bernoulli 24 (2018), no. 1, 231--256. doi:10.3150/16-BEJ872. https://projecteuclid.org/euclid.bj/1501142441

#### References

• [1] Adams, R.P., Murray, I. and MacKay, D.J. (2009). Tractable nonparametric Bayesian inference in Poisson processes with Gaussian process intensities. In Proceedings of the 26th Annual International Conference on Machine Learning. New York: ACM.
• [2] Ahmed, S. and Reid, N. (2001). Empirical Bayes and Likelihood Inference. Lecture Notes in Statistics 148. New York: Springer.
• [3] Andersen, P.K., Borgan, Ø., Gill, R.D. and Keiding, N. (1993). Statistical Models Based on Counting Processes. Springer Series in Statistics. New York: Springer.
• [4] Barron, A., Schervish, M.J. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536–561.
• [5] Belitser, E., Serra, P. and van Zanten, H. (2015). Rate-optimal Bayesian intensity smoothing for inhomogeneous Poisson processes. J. Statist. Plann. Inference 166 24–35.
• [6] Berger, J.O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer Series in Statistics. New York: Springer.
• [7] Carroll, R.J. and Hall, P. (1988). Optimal rates of convergence for deconvolving a density. J. Amer. Statist. Assoc. 83 1184–1186.
• [8] Casella, G. (1985). An introduction to empirical Bayes data analysis. Amer. Statist. 39 83–87.
• [9] Cheng, N. and Yuan, T. (2013). Nonparametric Bayesian lifetime data analysis using Dirichlet process lognormal mixture model. Naval Res. Logist. 60 208–221.
• [10] Clyde, M. and George, E.I. (2000). Flexible empirical Bayes estimation for wavelets. J. R. Stat. Soc. Ser. B. Stat. Methodol. 62 681–698.
• [11] Cui, W. and George, E.I. (2008). Empirical Bayes vs. fully Bayes variable selection. J. Statist. Plann. Inference 138 888–900.
• [12] Daley, D.J. and Vere-Jones, D. (2003). An Introduction to the Theory of Point Processes. Vol. I, 2nd ed. Probability and Its Applications (New York). New York: Springer.
• [13] Daley, D.J. and Vere-Jones, D. (2008). An Introduction to the Theory of Point Processes. Vol. II, 2nd ed. Probability and Its Applications (New York). New York: Springer.
• [14] Donnet, S., Rivoirard, V., Rousseau, J. and Scricciolo, C. (2014). Posterior concentration rates for empirical Bayes procedures, with applications to Dirichlet Process mixtures. Available at arXiv:1406.4406v1.
• [15] Donnet, S., Rivoirard, V., Rousseau, J. and Scricciolo, C. (2016). Supplement to “Posterior concentration rates for empirical Bayes procedures with applications to Dirichlet process mixtures.” DOI:10.3150/16-BEJ872SUPP.
• [16] Donnet, S., Rivoirard, V., Rousseau, J. and Scricciolo, C. (2016). Posterior concentration rates for counting processes with Aalen multiplicative intensities. Bayesian Anal. To appear.
• [17] Fall, M.D. and Barat, É. (2012). Gibbs sampling methods for Pitman-Yor mixture models. Technical report.
• [18] Fan, J. (1991). Global behavior of deconvolution kernel estimates. Statist. Sinica 1 541–551.
• [19] Fan, J. (1991). On the optimal rates of convergence for nonparametric deconvolution problems. Ann. Statist. 19 1257–1272.
• [20] Fan, J. (1992). Deconvolution with supersmooth distributions. Canad. J. Statist. 20 155–169.
• [21] Ferguson, T.S. (1974). Prior distributions on spaces of probability measures. Ann. Statist. 2 615–629.
• [22] Gaïffas, S. and Guilloux, A. (2012). High-dimensional additive hazards models and the Lasso. Electron. J. Stat. 6 522–546.
• [23] Ghosal, S., Ghosh, J.K. and van der Vaart, A.W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531.
• [24] Ghosal, S. and van der Vaart, A. (2007). Convergence rates of posterior distributions for non i.i.d. observations. Ann. Statist. 35 192–223.
• [25] Ghosal, S. and van der Vaart, A. (2007). Posterior convergence rates of Dirichlet mixtures at smooth densities. Ann. Statist. 35 697–723.
• [26] Ghosal, S. and van der Vaart, A.W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist. 29 1233–1263.
• [27] Ghosh, J.K. and Ramamoorthi, R.V. (2003). Bayesian Nonparametrics. Springer Series in Statistics. New York: Springer.
• [28] Hansen, N.R., Reynaud-Bouret, P. and Rivoirard, V. (2015). Lasso and probabilistic inequalities for multivariate point processes. Bernoulli 21 83–143.
• [29] Hjort, N.L., Holmes, C., Müller, P. and Walker, S.G., eds. (2010). Bayesian Nonparametrics. Cambridge Series in Statistical and Probabilistic Mathematics 28. Cambridge: Cambridge Univ. Press.
• [30] Karr, A.F. (1991). Point Processes and Their Statistical Inference, 2nd ed. Probability: Pure and Applied 7. New York: Dekker.
• [31] Kottas, A. and Sansó, B. (2007). Bayesian mixture modeling for spatial Poisson process intensities, with applications to extreme value analysis. J. Statist. Plann. Inference 137 3151–3163.
• [32] Kruijer, W., Rousseau, J. and van der Vaart, A. (2010). Adaptive Bayesian density estimation with location-scale mixtures. Electron. J. Stat. 4 1225–1257.
• [33] Lo, A.Y. (1992). Bayesian inference for Poisson process models with censored data. J. Nonparametr. Stat. 2 71–80.
• [34] McAuliffe, J.D., Blei, D.M. and Jordan, M.I. (2006). Nonparametric empirical Bayes for the Dirichlet process mixture model. Stat. Comput. 16 5–14.
• [35] Petrone, S., Rousseau, J. and Scricciolo, C. (2014). Bayes and empirical Bayes: Do they merge? Biometrika 101 285–302.
• [36] Reynaud-Bouret, P. (2006). Penalized projection estimators of the Aalen multiplicative intensity. Bernoulli 12 633–661.
• [37] Richardson, S. and Green, P.J. (1997). On Bayesian analysis of mixtures with an unknown number of components. J. R. Stat. Soc. Ser. B. Stat. Methodol. 59 731–792.
• [38] Rousseau, J. (2010). Rates of convergence for the posterior distributions of mixtures of betas and adaptive nonparametric estimation of the density. Ann. Statist. 38 146–180.
• [39] Rousseau, J. and Szabó, B.T. (2015). Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator. Technical report.
• [40] Salomond, J.-B. (2014). Concentration rate and consistency of the posterior distribution for selected priors under monotonicity constraints. Electron. J. Stat. 8 1380–1404.
• [41] Sarkar, A., Pati, D., Mallick, B.K. and Carroll, R.J. (2013). Adaptive posterior convergence rates in Bayesian density deconvolution with supersmooth errors. Technical report. Available at arXiv:1308.5427v2.
• [42] Schwartz, L. (1965). On Bayes procedures. Z. Wahrsch. Verw. Gebiete 4 10–26.
• [43] Scricciolo, C. (2014). Adaptive Bayesian density estimation in $L^{p}$-metrics with Pitman–Yor or normalized inverse-Gaussian process kernel mixtures. Bayesian Anal. 9 475–520.
• [44] Scricciolo, C. (2015). Empirical Bayes conditional density estimation. Statistica LXXV 37–55.
• [45] Serra, P. and Krivobokova, T. (2014). Adaptive empirical Bayesian smoothing splines. Technical report.
• [46] Shen, W., Tokdar, S.T. and Ghosal, S. (2013). Adaptive Bayesian multivariate density estimation with Dirichlet mixtures. Biometrika 100 623–640.
• [47] Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687–714.
• [48] Sniekers, S. and van der Vaart, A. (2015). Adaptive Bayesian credible sets in regression with a Gaussian process prior. Electron. J. Stat. 9 2475–2527.
• [49] Szabó, B., van der Vaart, A. and van Zanten, H. (2015). Honest Bayesian confidence sets for the ${L}^{2}$-norm. J. Statist. Plann. Inference 166 36–51.
• [50] Szabó, B., van der Vaart, A.W. and van Zanten, J.H. (2015). Frequentist coverage of adaptive nonparametric Bayesian credible sets. Ann. Statist. 43 1391–1428.
• [51] Szabó, B.T., van der Vaart, A.W. and van Zanten, J.H. (2013). Empirical Bayes scaling of Gaussian priors in the white noise model. Electron. J. Stat. 7 991–1018.
• [52] Williamson, R.E. (1956). Multiply monotone functions and their Laplace transforms. Duke Math. J. 23 189–207.

#### Supplemental materials

• Supplement to “Posterior concentration rates for empirical Bayes procedures with applications to Dirichlet process mixtures”. This supplement contains the proofs of Theorem 2, Proposition 1 and Theorem 3 from the mentioned article.