The Annals of Statistics

The Bernstein–von Mises theorem and nonregular models

Natalia A. Bochkina and Peter J. Green

Full-text: Open access

Abstract

We study the asymptotic behaviour of the posterior distribution in a broad class of statistical models where the “true” solution occurs on the boundary of the parameter space. We show that in this case Bayesian inference is consistent, and that the posterior distribution has not only Gaussian components as in the case of regular models (the Bernstein–von Mises theorem) but also has Gamma distribution components whose form depends on the behaviour of the prior distribution near the boundary and have a faster rate of convergence. We also demonstrate a remarkable property of Bayesian inference, that for some models, there appears to be no bound on efficiency of estimating the unknown parameter if it is on the boundary of the parameter space. We illustrate the results on a problem from emission tomography.

Article information

Source
Ann. Statist., Volume 42, Number 5 (2014), 1850-1878.

Dates
First available in Project Euclid: 11 September 2014

Permanent link to this document
https://projecteuclid.org/euclid.aos/1410440627

Digital Object Identifier
doi:10.1214/14-AOS1239

Mathematical Reviews number (MathSciNet)
MR3262470

Zentralblatt MATH identifier
1305.62112

Subjects
Primary: 62F12: Asymptotic properties of estimators 62F15: Bayesian inference

Keywords
Approximate posterior Bayesian inference Bernstein–von Mises theorem boundary nonregular posterior concentration SPECT tomography total variation distance variance estimation in mixed models

Citation

Bochkina, Natalia A.; Green, Peter J. The Bernstein–von Mises theorem and nonregular models. Ann. Statist. 42 (2014), no. 5, 1850--1878. doi:10.1214/14-AOS1239. https://projecteuclid.org/euclid.aos/1410440627


Export citation

References

  • Barron, A., Schervish, M. J. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536–561.
  • Bertsekas, D. P. (2003). Convex Analysis and Optimization. Athena Scientific and Tsinghua Univ. Press, Belmont, MA.
  • Besag, J. (1986). On the statistical analysis of dirty pictures. J. Roy. Statist. Soc. Ser. B 48 259–302.
  • Bochkina, N. (2013). Consistency of the posterior distribution in generalized linear inverse problems. Inverse Problems 29 095010, 43.
  • Chernozhukov, V. and Hong, H. (2004). Likelihood estimation and inference in a class of nonregular econometric models. Econometrica 72 1445–1480.
  • Douc, R., Moulines, E., Olsson, J. and van Handel, R. (2011). Consistency of the maximum likelihood estimator for general hidden Markov models. Ann. Statist. 39 474–513.
  • Dudley, R. M. and Haughton, D. (2002). Asymptotic normality with small relative errors of posterior probabilities of half-spaces. Ann. Statist. 30 1311–1344.
  • Erkanli, A. (1994). Laplace approximations for posterior expectations when the mode occurs at the boundary of the parameter space. J. Amer. Statist. Assoc. 89 250–258.
  • Geman, S. and Geman, D. (1984). Stochastic relaxation, gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6 721–741.
  • Ghosal, S., Ghosh, J. K. and Samanta, T. (1995). On convergence of posterior distributions. Ann. Statist. 23 2145–2152.
  • Ghosal, S. and Samanta, T. (1995). Asymptotic behaviour of Bayes estimates and posterior distributions in multiparameter nonregular cases. Math. Methods Statist. 4 361–388.
  • Ghosh, J. K., Ghosal, S. and Samanta, T. (1994). Stability and convergence of the posterior in non-regular problems. In Statistical Decision Theory and Related Topics, V (West Lafayette, IN, 1992) (S. S. Gupta and J. O. Berger, eds.) 183–199. Springer, New York.
  • Green, P. J. (1990). Bayesian reconstructions from emission tomography data using a modified EM algorithm. IEEE Trans. Med. Imag. 9 84–93.
  • Hirano, K. and Porter, J. R. (2003). Asymptotic efficiency in parametric structural models with parameter-dependent support. Econometrica 71 1307–1338.
  • Ibragimov, I. A. and Has’minskiĭ, R. Z. (1981). Statistical Estimation: Asymptotic Theory. Springer, New York.
  • Johnstone, I. M. and Silverman, B. W. (1990). Speed of estimation in positron emission tomography and related inverse problems. Ann. Statist. 18 251–280.
  • LeCam, L. (1953). On some asymptotic properties of maximum likelihood estimates and related Bayes’ estimates. Univ. California Publ. Statist. 1 277–329.
  • Le Cam, L. and Yang, G. L. (1990). Asymptotics in Statistics: Some Basic Concepts. Springer, New York.
  • Moran, P. A. P. (1971). Maximum-likelihood estimation in non-standard conditions. Proc. Cambridge Philos. Soc. 70 441–450.
  • Nelder, J. A. and Wedderburn, R. W. M. (1972). Generalized linear models. J. R. Stat. Soc. A, General 135 370–384.
  • Petrone, S., Rousseau, J. and Scricciolo, C. (2012). Bayes and empirical Bayes: Do they merge? Biometrika 99 1–21.
  • Polyak, B. T. (1983). Introduction to Optimization (Vvedenie v Optimizatsiyu, in Russian). Nauka, Moscow.
  • Self, S. G. and Liang, K.-Y. (1987). Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. J. Amer. Statist. Assoc. 82 605–610.
  • Shapiro, A. (2000). On the asymptotics of constrained local $M$-estimators. Ann. Statist. 28 948–960.
  • Spokoiny, V. (2012). Parametric estimation. Finite sample theory. Ann. Statist. 40 2877–2909.
  • van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Univ. Press, Cambridge.
  • Vu, H. T. V. and Zhou, S. (1997). Generalization of likelihood ratio tests under nonstandard conditions. Ann. Statist. 25 897–916.
  • Weir, I. S. (1997). Fully Bayesian reconstructions from single-photon emission computed tomography data. J. Amer. Statist. Assoc. 92 49–60.