Bayesian Analysis

Posterior Concentration Rates for Infinite Dimensional Exponential Families

Vincent Rivoirard and Judith Rousseau

Full-text: Open access

Abstract

In this paper we derive adaptive non-parametric rates of concentration of the posterior distributions for the density model on the class of Sobolev and Besov spaces. For this purpose, we build prior models based on wavelet or Fourier expansions of the logarithm of the density. The prior models are not necessarily Gaussian.

Article information

Source
Bayesian Anal. Volume 7, Number 2 (2012), 311-334.

Dates
First available: 16 June 2012

Permanent link to this document
http://projecteuclid.org/euclid.ba/1339878890

Digital Object Identifier
doi:10.1214/12-BA710

Mathematical Reviews number (MathSciNet)
MR2934953

Citation

Rivoirard, Vincent; Rousseau, Judith. Posterior Concentration Rates for Infinite Dimensional Exponential Families. Bayesian Analysis 7 (2012), no. 2, 311--334. doi:10.1214/12-BA710. http://projecteuclid.org/euclid.ba/1339878890.


Export citation

References

  • Barron, A., Schervish, M., and Wasserman, L. (1999). “The consistency of posterior distributions in nonparametric problems.” Annals of Statistics, 27: 536–561.
  • Bergh, J. and Löfström, J. (1976). Interpolation spaces. An introduction. Berlin: Springer-Verlag. Grundlehren der Mathematischen Wissenschaften, No. 223.
  • Castillo, I. (2008). “Lower bounds for posterior rates with Gaussian process priors.” Electronic Journal of Statistics, 2: 1281–1299.
  • Dalalyan, A. S. and Tsybakov, A. B. (2007). “Aggregation by exponential weighting and sharp oracle inequalities.” In Learning theory, volume 4539 of Lecture Notes in Computer Science, 97–111. Berlin: Springer. URL http://dx.doi.org/10.1007/978-3-540-72927-3_9
  • DeVore, R. A. and Lorentz, G. G. (1993). Constructive approximation, volume 303 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Berlin: Springer-Verlag.
  • Diaconis, P. and Freedman, D. (1986). “On the consistency of Bayes estimates.” Annals of Statistics, 14: 1–26.
  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G., and Picard, D. (1995). “Wavelet shrinkage: asymptopia?” Journal of the Royal Statistical Society Series B, 57(2): 301–369. With discussion and a reply by the authors. URL http://links.jstor.org/sici?sici=0035-9246(1995)57:2<301:WSA>2.0.CO;2-S&origin=MSN
  • Ghosal, S., Ghosh, J. K., and van der Vaart, A. (2000). “Convergence rates of posterior distributions.” Annals of Statistics, 28: 500–531.
  • Ghosal, S., Lember, J., and van der Vaart, A. (2008). “Nonparametric Bayesian model selection and averaging.” Electronic Journal of Statistics, 2: 63–89.
  • Härdle, W., Kerkyacharian, G., Picard, D., and Tsybakov, A. (1998). Wavelets, approximation, and statistical applications. Lecture Notes in Statistics, 129. New York: Springer-Verlag.
  • Huang, T. (2004). “Convergence rates for posterior distributions and adaptive estimation.” Annals of Statistics, 32: 1556–1593.
  • Johnstone, I. M. (1994). “Minimax Bayes, asymptotic minimax and sparse wavelet priors.” In Statistical decision theory and related topics, V (West Lafayette, IN, 1992), 303–326. New York: Springer.
  • Mallat, S. (1998). A wavelet tour of signal processing. San Diego, CA: Academic Press Inc.
  • Meyer, Y. (1992). Wavelets and operators, volume 37 of Cambridge Studies in Advanced Mathematics. Cambridge: Cambridge University Press. Translated from the 1990 French original by D. H. Salinger.
  • Park, T. and Casella, G. (2008). “The Bayesian Lasso.” Journal of the American Statistical Association, 103(482): 681–686. URL http://dx.doi.org/10.1198/016214508000000337
  • Reynaud-Bouret, P., Rivoirard, V., and Tuleau-Malot, C. (2011). “Adaptive density estimation: a curse of support?” Journal of Statistical Planning and Inference, 141(1): 115–139. URL http://dx.doi.org/10.1016/j.jspi.2010.05.017
  • Rivoirard, V. (2006). “Nonlinear estimation over weak Besov spaces and minimax Bayes method.” Bernoulli, 12(4): 609–632. URL http://dx.doi.org/10.3150/bj/1155735929
  • Scricciolo, C. (2006). “Convergence rates for Bayesian density estimation of infinite-dimensional exponential families.” Annals of Statistics, 34: 2897–2920.
  • Shen, X. and Wasserman, L. (2001). “Rates of convergence of posterior distributions.” Annals of Statistics, 29: 687–714.
  • Tsybakov, A. B. (2009). Introduction to nonparametric estimation. Springer Series in Statistics. New York: Springer. Revised and extended from the 2004 French original, Translated by Vladimir Zaiats.
  • van der Vaart, A. W. and van Zanten, J. H. (2008). “Rates of contraction of posterior distributions based on Gaussian process priors.” Annals of Statistics, 36(3): 1435–1463.
  • Verdinelli, I. and Wasserman, L. (1998). “Bayesian goodness-of-fit testing using infinite-dimensional exponential families.” Annals of Statistics, 26: 1215–1241.
  • Wong, W. and Shen, X. (1995). “Probability inequalities for likelihood ratios and convergence rates of sieve MLEs.” Annals of Statistics, 23: 339–362.
  • Zhao, L. (2000). “Bayesian aspects of some nonparametric problems.” Annals of Statistics, 28: 532–552.