The Annals of Statistics

Bernstein–von Mises theorems for Gaussian regression with increasing number of regressors

Dominique Bontemps

Full-text: Open access

Abstract

This paper brings a contribution to the Bayesian theory of nonparametric and semiparametric estimation. We are interested in the asymptotic normality of the posterior distribution in Gaussian linear regression models when the number of regressors increases with the sample size. Two kinds of Bernstein–von Mises theorems are obtained in this framework: nonparametric theorems for the parameter itself, and semiparametric theorems for functionals of the parameter. We apply them to the Gaussian sequence model and to the regression of functions in Sobolev and Cα classes, in which we get the minimax convergence rates. Adaptivity is reached for the Bayesian estimators of functionals in our applications.

Article information

Source
Ann. Statist., Volume 39, Number 5 (2011), 2557-2584.

Dates
First available in Project Euclid: 30 November 2011

Permanent link to this document
https://projecteuclid.org/euclid.aos/1322663468

Digital Object Identifier
doi:10.1214/11-AOS912

Mathematical Reviews number (MathSciNet)
MR2906878

Zentralblatt MATH identifier
1231.62061

Subjects
Primary: 62F15: Bayesian inference 62J05: Linear regression 62G20: Asymptotic properties

Keywords
Nonparametric Bayesian statistics semiparametric Bayesian statistics Bernstein–von Mises theorem posterior asymptotic normality adaptive estimation

Citation

Bontemps, Dominique. Bernstein–von Mises theorems for Gaussian regression with increasing number of regressors. Ann. Statist. 39 (2011), no. 5, 2557--2584. doi:10.1214/11-AOS912. https://projecteuclid.org/euclid.aos/1322663468


Export citation

References

  • Bontemps, D. (2011). Supplement to “Bernstein–von Mises theorems for Gaussian regression with increasing number of regressors.” DOI:10.1214/11-AOS912SUPP.
  • Boucheron, S. and Gassiat, E. (2009). A Bernstein–von Mises theorem for discrete probability distributions. Electron. J. Stat. 3 114–148.
  • Castillo, I. (2010). A semi-parametric Bernstein–von Mises theorem. Probab. Theory Related Fields. DOI:10.1007/s00440-010-0316-5.
  • Clarke, B. S. and Barron, A. R. (1990). Information-theoretic asymptotics of Bayes methods. IEEE Trans. Inform. Theory 36 453–471.
  • Clarke, B. and Ghosal, S. (2010). Reference priors for exponential families with increasing dimension. Electron. J. Stat. 4 737–780.
  • de Boor, C. (1978). A Practical Guide to Splines. Applied Mathematical Sciences 27. Springer, New York.
  • Freedman, D. (1999). Wald lecture: On the Bernstein–von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119–1140.
  • Ghosal, S. (1999). Asymptotic normality of posterior distributions in high-dimensional linear models. Bernoulli 5 315–331.
  • Ghosal, S. (2000). Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity. J. Multivariate Anal. 74 49–68.
  • Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531.
  • Ghosal, S. and van der Vaart, A. (2007). Convergence rates of posterior distributions for non-i.i.d. observations. Ann. Statist. 35 192–223.
  • Kim, Y. (2006). The Bernstein–von Mises theorem for the proportional hazard model. Ann. Statist. 34 1678–1700.
  • Kim, Y. and Lee, J. (2004). A Bernstein–von Mises theorem in the nonparametric right-censoring model. Ann. Statist. 32 1492–1512.
  • Massart, P. (2007). Concentration Inequalities and Model Selection. Lecture Notes in Math. 1896. Springer, Berlin.
  • Rivoirard, V. and Rousseau, J. (2009). Bernstein von Mises theorem for linear functionals of the density. Available at http://arxiv.org/abs/0908.4167.
  • Shen, X. (2002). Asymptotic normality of semiparametric and nonparametric posterior distributions. J. Amer. Statist. Assoc. 97 222–235.
  • Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687–714.
  • Tsybakov, A. B. (2004). Introduction à L’estimation Non-Paramétrique. Springer, Berlin.
  • van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Univ. Press, Cambridge.

Supplemental materials

  • Supplementary material: Supplement to “Bernstein–von Mises theorems for Gaussian regression with increasing number of regressors”. This contains the proofs of various technical results stated in the main article “Bernstein–von Mises Theorems for Gaussian regression with increasing number of regressors.”.