Bayesian Analysis

Finite Sample Bernstein – von Mises Theorem for Semiparametric Problems

Maxim Panov and Vladimir Spokoiny

Full-text: Open access

Abstract

The classical parametric and semiparametric Bernstein – von Mises (BvM) results are reconsidered in a non-classical setup allowing finite samples and model misspecification. In the case of a finite dimensional nuisance parameter we obtain an upper bound on the error of Gaussian approximation of the posterior distribution for the target parameter which is explicit in the dimension of the nuisance and target parameters. This helps to identify the so called critical dimension pn of the full parameter for which the BvM result is applicable. In the important i.i.d. case, we show that the condition “pn3/n is small” is sufficient for the BvM result to be valid under general assumptions on the model. We also provide an example of a model with the phase transition effect: the statement of the BvM theorem fails when the dimension pn approaches n1/3 . The results are extended to the case of infinite dimensional parameters with the nuisance parameter from a Sobolev class.

Article information

Source
Bayesian Anal., Volume 10, Number 3 (2015), 665-710.

Dates
First available in Project Euclid: 2 February 2015

Permanent link to this document
https://projecteuclid.org/euclid.ba/1422884986

Digital Object Identifier
doi:10.1214/14-BA926

Mathematical Reviews number (MathSciNet)
MR3420819

Zentralblatt MATH identifier
1335.62057

Keywords
prior posterior Bayesian inference semiparametric critical dimension

Citation

Panov, Maxim; Spokoiny, Vladimir. Finite Sample Bernstein – von Mises Theorem for Semiparametric Problems. Bayesian Anal. 10 (2015), no. 3, 665--710. doi:10.1214/14-BA926. https://projecteuclid.org/euclid.ba/1422884986


Export citation

References

  • Barron, A., Schervish, M. J., and Wasserman, L. (1996). “The Consistency of Posterior Distributions in Nonparametric Problems.” The Annals of Statistics, 27: 536–561.
  • Bickel, P. J. and Kleijn, B. J. K. (2012). “The semiparametric Bernstein-von Mises theorem.” The Annals of Statistics, 40(1): 206–237.
  • Bochkina, N. and Green, P. J. (2014). “The Bernstein-von Mises theorem and non-regular models.” The Annals of Statistics, 42(5): 1850–1878.
  • Bontemps, D. (2011). “Bernstein–von Mises theorem for Gaussian regression with increasing number of regressors.” The Annals of Statistics, 39(5): 2557–2584.
  • Boucheron, S. and Gassiat, E. (2009). “A Bernstein-von Mises theorem for discrete probability distributions.” Electronic Journal of Statistics, 3: 114–148.
  • Boucheron, S. and Massart, P. (2011). “A high-dimensional Wilks phenomenon.” Probability Theory and Related Fields, 150: 405–433.
  • Castillo, I. (2012). “A semiparametric Bernstein–von Mises theorem for Gaussian process priors.” Probability Theory and Related Fields, 152: 53–99.
  • Castillo, I. and Nickl, R. (2013). “Nonparametric Bernstein–von Mises theorems in Gaussian white noise.” The Annals of Statistics, 41(4): 1999–2028.
  • Castillo, I. and Rousseau, J. (2013). “A General Bernstein–von Mises Theorem in semiparametric models.” Available at arXiv:1305.4482 [math.ST].
  • Cheng, G. and Kosorok, M. R. (2008). “General frequentist properties of the posterior profile distribution.” The Annals of Statistics, 36(4): 1819–1853.
  • Chernozhukov, V. and Hong, H. (2003). “An MCMC approach to classical estimation.” Journal of Econometrics, 115(2): 293–346.
  • Cox, D. D. (1993). “An analysis of Bayesian inference for nonparametric regression.” The Annals of Statistics, 21(2): 903–923.
  • Freedman, D. (1999). “On the Bernstein-von Mises theorem with infinite-dimensional parameters.” The Annals of Statistics, 27(4): 1119–1140.
  • Ghosal, S. (1999). “Asymptotic normality of posterior distributions in high-dimensional linear models.” Bernoulli, 5(2): 315–331.
  • — (2000). “Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity.” Journal of Multivariate Analysis, 74(1): 49–68.
  • Ibragimov, I. and Khas’minskij, R. (1981). Statistical estimation. Asymptotic theory. Translated from the Russian by Samuel Kotz. New York–Heidelberg–Berlin: Springer-Verlag.
  • Johnstone, I. M. (2010). “High dimensional Bernstein–von Mises: simple examples.” In Borrowing strength: theory powering applications—a Festschrift for Lawrence D. Brown, volume 6 of Institute of Mathematical Statistics Collections, 87–98. Beachwood, OH: Institute of Mathematical Statistics.
  • Kim, Y. (2006). “The Bernstein–von Mises theorem for the proportional hazard model.” The Annals of Statistics, 34(4): 1678–1700.
  • Kim, Y. and Lee, J. (2004). “A Bernstein–von Mises theorem in the nonparametric right-censoring model.” The Annals of Statistics, 32(4): 1492–1512.
  • Kleijn, B. J. K. and van der Vaart, A. W. (2006). “Misspecification in infinite-dimensional Bayesian statistics.” The Annals of Statistics, 34(2): 837–877.
  • — (2012). “The Bernstein-von-Mises theorem under misspecification.” Electronic Journal of Statistics, 6: 354–381.
  • Le Cam, L. and Yang, G. L. (1990). Asymptotics in Statistics: Some Basic Concepts. Springer in Statistics.
  • Leahu, H. (2011). “On the Bernstein-von Mises phenomenon in the Gaussian white noise model.” Electronic Journal of Statistics, 5: 373–404.
  • McCullagh, P. and Nelder, J. (1989). Generalized linear models. 2nd ed.. Monographs on Statistics and Applied Probability. 37. London etc.: Chapman and Hall.
  • Polzehl, J. and Spokoiny, V. (2006). “Propagation-separation approach for local likelihood estimation.” Probability Theory and Related Fields, 135(3): 335–362.
  • Rivoirard, V. and Rousseau, J. (2012). “Bernstein–von Mises theorem for linear functionals of the density.” The Annals of Statistics, 40(3): 1489–1523.
  • Schwartz, L. (1965). “On Bayes Procedures.” Probability Theory and Related Fields, 4(1): 10–26.
  • Shen, X. (2002). “Asymptotic normality of semiparametric and nonparametric posterior distributions.” Journal of the American Statistical Association, 97(457): 222–235.
  • Spokoiny, V. (2012). “Parametric estimation. Finite sample theory.” The Annals of Statistics, 40(6): 2877–2909.
  • van der Vaart, A. W. (2000). Asymptotic Statistics (Cambridge Series in Statistical and Probabilistic Mathematics). Cambridge University Press.