Institute of Mathematical Statistics Collections

High dimensional Bernstein-von Mises: simple examples

Iain M. Johnstone

Full-text: Open access

Abstract

In Gaussian sequence models with Gaussian priors, we develop some simple examples to illustrate three perspectives on matching of posterior and frequentist probabilities when the dimension p increases with sample size n: (i) convergence of joint posterior distributions, (ii) behavior of a non-linear functional: squared error loss, and (iii) estimation of linear functionals. The three settings are progressively less demanding in terms of conditions needed for validity of the Bernstein-von Mises theorem.

Chapter information

Source
James O. Berger, T. Tony Cai and Iain M. Johnstone, eds., Borrowing Strength: Theory Powering Applications – A Festschrift for Lawrence D. Brown (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2010), 87-98

Dates
First available in Project Euclid: 26 October 2010

Permanent link to this document
https://projecteuclid.org/euclid.imsc/1288099014

Digital Object Identifier
doi:10.1214/10-IMSCOLL607

Subjects
Primary: 62E20: Asymptotic distribution theory
Secondary: 62F15: Bayesian inference

Keywords
high dimensional inference Gaussian sequence linear functional squared error loss posterior distribution frequentist

Rights
Copyright © 2010, Institute of Mathematical Statistics

Citation

Johnstone, Iain M. High dimensional Bernstein-von Mises: simple examples. Borrowing Strength: Theory Powering Applications – A Festschrift for Lawrence D. Brown, 87--98, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2010. doi:10.1214/10-IMSCOLL607. https://projecteuclid.org/euclid.imsc/1288099014


Export citation

References

  • [1] Borwanker, J., Kallianpur, G. and Prakasa Rao, B. L. S. (1971). The Bernstein-von Mises theorem for Markov processes. Ann. Math. Statist. 42 1241–1253.
  • [2] Boucheron, S. and Gassiat, E. (2009). A Bernstein-von Mises theorem for discrete probability distributions. Electron. J. Stat. 3 114–148.
  • [3] Castillo, I. (2008). A semiparametric Bernstein-von Mises theorem. Submitted.
  • [4] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903–923.
  • [5] Dasgupta, A. and Lahiri, S. N. (2010). Density estimation in high and ultra high dimensions, regularization, and the L1 asymptotics. In A Festschrift for William Strawderman (D. Fourdrinier and É. Marchand and A. Rukhin, eds.). IMS.
  • [6] Freedman, D. (1999). On the Bernstein-von Mises theorem with infinite dimensional parameters. Ann. Statist. 27 1119–1140.
  • [7] Ghosal, S. (1997). Normal approximation to the posterior distribution for generalized linear models with many covariates. Math. Methods Statist. 6 332–348.
  • [8] Ghosal, S. (1999). Asymptotic normality of posterior distributions in high-dimensional linear models. Bernoulli 5 315–331.
  • [9] Ghosal, S. (2000). Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity. J. Multivariate Anal. 74 49–68.
  • [10] Ghosal, S. (2010). The Dirichlet process, related priors and posterior asymptotics. In Bayesian Nonparametrics (N. L. Hjort, C. Holmes, P. Müller and S. G. Walker, eds.), Chapter 2. Cambridge Univ. Press.
  • [11] Ghosal, S. and van der Vaart, A. (2010). Theory of Nonparametric Bayesian Inference. Cambridge Univ. Press. In preparation.
  • [12] Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics. Springer Series in Statistics. Springer, New York.
  • [13] Heyde, C. C. and Johnstone, I. M. (1979). On asymptotic posterior normality for stochastic processes. J. Roy. Statist. Soc. Ser. B 41 184–189.
  • [14] Johnstone, I. M. (2010). Function estimation and Gaussian sequence models. Book manuscript at www-stat.stanford.edu.
  • [15] Kim, Y. (2006). The Bernstein-von Mises theorem for the proportional hazard model. Ann. Statist. 34 1678–1700.
  • [16] Kim, Y. and Lee, J. (2004). A Bernstein-von Mises theorem in the nonparametric right-censoring model. Ann. Statist. 32 1492–1512.
  • [17] Lehmann, E. L. and Casella, G. (1998). Theory of Point Estimation, 2nd ed. Springer Texts in Statistics. Springer, New York.
  • [18] Pinsker, M. (1980). Optimal filtering of square integrable signals in Gaussian white noise. Problems of Information Transmission 16 120–133. Originally in Russian in Problemy Peredatsii Informatsii 16 52–68.
  • [19] Rivoirard, V. and Rousseau, J. (2009). Bernstein von Mises theorem for linear functionals of the density. Submitted.
  • [20] Shen, X. (2002). Asymptotic normality of semiparametric and nonparametric posterior distributions. J. Amer. Statist. Assoc. 97 222–235.
  • [21] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge.