Finite Sample Bernstein – von Mises Theorem for Semiparametric Problems

Maxim Panov; Vladimir Spokoiny

doi:10.1214/14-BA926

Bayesian Analysis

Bayesian Anal.
Volume 10, Number 3 (2015), 665-710.

Finite Sample Bernstein – von Mises Theorem for Semiparametric Problems

Maxim Panov and Vladimir Spokoiny

Full-text: Open access

Enhanced PDF (483 KB)

Abstract

The classical parametric and semiparametric Bernstein – von Mises (BvM) results are reconsidered in a non-classical setup allowing finite samples and model misspecification. In the case of a finite dimensional nuisance parameter we obtain an upper bound on the error of Gaussian approximation of the posterior distribution for the target parameter which is explicit in the dimension of the nuisance and target parameters. This helps to identify the so called critical dimension $p_{n}$ of the full parameter for which the BvM result is applicable. In the important i.i.d. case, we show that the condition “ $p_{n}^{3}/n$ is small” is sufficient for the BvM result to be valid under general assumptions on the model. We also provide an example of a model with the phase transition effect: the statement of the BvM theorem fails when the dimension $p_{n}$ approaches $n^{1/3}$ . The results are extended to the case of infinite dimensional parameters with the nuisance parameter from a Sobolev class.

Article information

Source
Bayesian Anal., Volume 10, Number 3 (2015), 665-710.

Dates
First available in Project Euclid: 2 February 2015

Permanent link to this document
https://projecteuclid.org/euclid.ba/1422884986

Digital Object Identifier
doi:10.1214/14-BA926

Mathematical Reviews number (MathSciNet)
MR3420819

Zentralblatt MATH identifier
1335.62057

Keywords
prior posterior Bayesian inference semiparametric critical dimension

Citation

Panov, Maxim; Spokoiny, Vladimir. Finite Sample Bernstein – von Mises Theorem for Semiparametric Problems. Bayesian Anal. 10 (2015), no. 3, 665--710. doi:10.1214/14-BA926. https://projecteuclid.org/euclid.ba/1422884986

References

Barron, A., Schervish, M. J., and Wasserman, L. (1996). “The Consistency of Posterior Distributions in Nonparametric Problems.” The Annals of Statistics, 27: 536–561.
Mathematical Reviews (MathSciNet): MR1714718
Zentralblatt MATH: 0980.62039
Digital Object Identifier: doi:10.1214/aos/1017939142
Project Euclid: euclid.aos/1018031206
Bickel, P. J. and Kleijn, B. J. K. (2012). “The semiparametric Bernstein-von Mises theorem.” The Annals of Statistics, 40(1): 206–237.
Mathematical Reviews (MathSciNet): MR3013185
Zentralblatt MATH: 1246.62081
Digital Object Identifier: doi:10.1214/11-AOS921
Project Euclid: euclid.aos/1333029963
Bochkina, N. and Green, P. J. (2014). “The Bernstein-von Mises theorem and non-regular models.” The Annals of Statistics, 42(5): 1850–1878.
Mathematical Reviews (MathSciNet): MR3262470
Zentralblatt MATH: 1305.62112
Digital Object Identifier: doi:10.1214/14-AOS1239
Project Euclid: euclid.aos/1410440627
Bontemps, D. (2011). “Bernstein–von Mises theorem for Gaussian regression with increasing number of regressors.” The Annals of Statistics, 39(5): 2557–2584.
Mathematical Reviews (MathSciNet): MR2906878
Zentralblatt MATH: 1231.62061
Digital Object Identifier: doi:10.1214/11-AOS912
Project Euclid: euclid.aos/1322663468
Boucheron, S. and Gassiat, E. (2009). “A Bernstein-von Mises theorem for discrete probability distributions.” Electronic Journal of Statistics, 3: 114–148.
Mathematical Reviews (MathSciNet): MR2471588
Zentralblatt MATH: 06166435
Digital Object Identifier: doi:10.1214/08-EJS262
Project Euclid: euclid.ejs/1233176792
Boucheron, S. and Massart, P. (2011). “A high-dimensional Wilks phenomenon.” Probability Theory and Related Fields, 150: 405–433.
Mathematical Reviews (MathSciNet): MR2824862
Zentralblatt MATH: 1230.62072
Digital Object Identifier: doi:10.1007/s00440-010-0278-7
Castillo, I. (2012). “A semiparametric Bernstein–von Mises theorem for Gaussian process priors.” Probability Theory and Related Fields, 152: 53–99.
Mathematical Reviews (MathSciNet): MR2875753
Zentralblatt MATH: 1232.62054
Digital Object Identifier: doi:10.1007/s00440-010-0316-5
Castillo, I. and Nickl, R. (2013). “Nonparametric Bernstein–von Mises theorems in Gaussian white noise.” The Annals of Statistics, 41(4): 1999–2028.
Mathematical Reviews (MathSciNet): MR3127856
Zentralblatt MATH: 1285.62052
Digital Object Identifier: doi:10.1214/13-AOS1133
Project Euclid: euclid.aos/1382547511
Castillo, I. and Rousseau, J. (2013). “A General Bernstein–von Mises Theorem in semiparametric models.” Available at arXiv:1305.4482 [math.ST].
Zentralblatt MATH: 05774394
Cheng, G. and Kosorok, M. R. (2008). “General frequentist properties of the posterior profile distribution.” The Annals of Statistics, 36(4): 1819–1853.
Mathematical Reviews (MathSciNet): MR2435457
Zentralblatt MATH: 1142.62031
Digital Object Identifier: doi:10.1214/07-AOS536
Project Euclid: euclid.aos/1216237301
Chernozhukov, V. and Hong, H. (2003). “An MCMC approach to classical estimation.” Journal of Econometrics, 115(2): 293–346.
Mathematical Reviews (MathSciNet): MR1984779
Zentralblatt MATH: 1043.62022
Digital Object Identifier: doi:10.1016/S0304-4076(03)00100-3
Cox, D. D. (1993). “An analysis of Bayesian inference for nonparametric regression.” The Annals of Statistics, 21(2): 903–923.
Mathematical Reviews (MathSciNet): MR1232525
Zentralblatt MATH: 0778.62003
Digital Object Identifier: doi:10.1214/aos/1176349157
Project Euclid: euclid.aos/1176349157
Freedman, D. (1999). “On the Bernstein-von Mises theorem with infinite-dimensional parameters.” The Annals of Statistics, 27(4): 1119–1140.
Mathematical Reviews (MathSciNet): MR1740119
Zentralblatt MATH: 0957.62002
Project Euclid: euclid.aos/1017938917
Ghosal, S. (1999). “Asymptotic normality of posterior distributions in high-dimensional linear models.” Bernoulli, 5(2): 315–331.
Mathematical Reviews (MathSciNet): MR1681701
Digital Object Identifier: doi:10.2307/3318438
Project Euclid: euclid.bj/1173147909
— (2000). “Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity.” Journal of Multivariate Analysis, 74(1): 49–68.
Mathematical Reviews (MathSciNet): MR1790613
Zentralblatt MATH: 1118.62309
Digital Object Identifier: doi:10.1006/jmva.1999.1874
Ibragimov, I. and Khas’minskij, R. (1981). Statistical estimation. Asymptotic theory. Translated from the Russian by Samuel Kotz. New York–Heidelberg–Berlin: Springer-Verlag.
Mathematical Reviews (MathSciNet): MR620321
Johnstone, I. M. (2010). “High dimensional Bernstein–von Mises: simple examples.” In Borrowing strength: theory powering applications—a Festschrift for Lawrence D. Brown, volume 6 of Institute of Mathematical Statistics Collections, 87–98. Beachwood, OH: Institute of Mathematical Statistics.
Mathematical Reviews (MathSciNet): MR2798513
Kim, Y. (2006). “The Bernstein–von Mises theorem for the proportional hazard model.” The Annals of Statistics, 34(4): 1678–1700.
Mathematical Reviews (MathSciNet): MR2283713
Zentralblatt MATH: 1246.62050
Digital Object Identifier: doi:10.1214/009053606000000533
Project Euclid: euclid.aos/1162567629
Kim, Y. and Lee, J. (2004). “A Bernstein–von Mises theorem in the nonparametric right-censoring model.” The Annals of Statistics, 32(4): 1492–1512.
Mathematical Reviews (MathSciNet): MR2089131
Zentralblatt MATH: 1047.62043
Digital Object Identifier: doi:10.1214/009053604000000175
Project Euclid: euclid.aos/1091626176
Kleijn, B. J. K. and van der Vaart, A. W. (2006). “Misspecification in infinite-dimensional Bayesian statistics.” The Annals of Statistics, 34(2): 837–877.
Mathematical Reviews (MathSciNet): MR2283395
Zentralblatt MATH: 1095.62031
Digital Object Identifier: doi:10.1214/009053606000000029
Project Euclid: euclid.aos/1151418243
— (2012). “The Bernstein-von-Mises theorem under misspecification.” Electronic Journal of Statistics, 6: 354–381.
Mathematical Reviews (MathSciNet): MR2988412
Digital Object Identifier: doi:10.1214/12-EJS675
Project Euclid: euclid.ejs/1332162333
Le Cam, L. and Yang, G. L. (1990). Asymptotics in Statistics: Some Basic Concepts. Springer in Statistics.
Mathematical Reviews (MathSciNet): MR1066869
Leahu, H. (2011). “On the Bernstein-von Mises phenomenon in the Gaussian white noise model.” Electronic Journal of Statistics, 5: 373–404.
Mathematical Reviews (MathSciNet): MR2802048
Zentralblatt MATH: 1274.62290
Digital Object Identifier: doi:10.1214/11-EJS611
Project Euclid: euclid.ejs/1305034907
McCullagh, P. and Nelder, J. (1989). Generalized linear models. 2nd ed.. Monographs on Statistics and Applied Probability. 37. London etc.: Chapman and Hall.
Mathematical Reviews (MathSciNet): MR3223057
Polzehl, J. and Spokoiny, V. (2006). “Propagation-separation approach for local likelihood estimation.” Probability Theory and Related Fields, 135(3): 335–362.
Mathematical Reviews (MathSciNet): MR2240690
Zentralblatt MATH: 1089.62033
Digital Object Identifier: doi:10.1007/s00440-005-0464-1
Rivoirard, V. and Rousseau, J. (2012). “Bernstein–von Mises theorem for linear functionals of the density.” The Annals of Statistics, 40(3): 1489–1523.
Mathematical Reviews (MathSciNet): MR3015033
Zentralblatt MATH: 1257.62036
Digital Object Identifier: doi:10.1214/12-AOS1004
Project Euclid: euclid.aos/1346850063
Schwartz, L. (1965). “On Bayes Procedures.” Probability Theory and Related Fields, 4(1): 10–26.
Mathematical Reviews (MathSciNet): MR184378
Digital Object Identifier: doi:10.1007/BF00535479
Shen, X. (2002). “Asymptotic normality of semiparametric and nonparametric posterior distributions.” Journal of the American Statistical Association, 97(457): 222–235.
Mathematical Reviews (MathSciNet): MR1947282
Zentralblatt MATH: 1073.62517
Digital Object Identifier: doi:10.1198/016214502753479365
Spokoiny, V. (2012). “Parametric estimation. Finite sample theory.” The Annals of Statistics, 40(6): 2877–2909.
Mathematical Reviews (MathSciNet): MR3097963
Zentralblatt MATH: 1296.62051
Digital Object Identifier: doi:10.1214/12-AOS1054
Project Euclid: euclid.aos/1360332187
van der Vaart, A. W. (2000). Asymptotic Statistics (Cambridge Series in Statistical and Probabilistic Mathematics). Cambridge University Press.
Mathematical Reviews (MathSciNet): MR1652247
Zentralblatt MATH: 0910.62001

Project Euclid

mathematics and statistics online

Bayesian Analysis

Finite Sample Bernstein – von Mises Theorem for Semiparametric Problems

More by Maxim Panov

More by Vladimir Spokoiny

Abstract

Article information

Citation

References

International Society for Bayesian Analysis

New content alerts

More like this