The Annals of Statistics

Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution

Eduard Belitser and Subhashis Ghosal

Full-text: Open access

Abstract

We consider the problem of estimating the mean of an infinite-break dimensional normal distribution from the Bayesian perspective. Under the assumption that the unknown true mean satisfies a "smoothness condition," we first derive the convergence rate of the posterior distribution for a prior that is the infinite product of certain normal distributions and compare with the minimax rate of convergence for point estimators. Although the posterior distribution can achieve the optimal rate of convergence, the required prior depends on a "smoothness parameter" q. When this parameter q is unknown, besides the estimation of the mean, we encounter the problem of selecting a model. In a Bayesian approach, this uncertainty in the model selection can be handled simply by further putting a prior on the index of the model. We show that if q takes values only in a discrete set, the resulting hierarchical prior leads to the same convergence rate of the posterior as if we had a single model. A slightly weaker result is presented when q is unrestricted. An adaptive point estimator based on the posterior distribution is also constructed.

Article information

Source
Ann. Statist., Volume 31, Number 2 (2003), 536-559.

Dates
First available in Project Euclid: 22 April 2003

Permanent link to this document
https://projecteuclid.org/euclid.aos/1051027880

Digital Object Identifier
doi:10.1214/aos/1051027880

Mathematical Reviews number (MathSciNet)
MR1983541

Zentralblatt MATH identifier
1039.62039

Subjects
Primary: 62G20: Asymptotic properties
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures 62G05: Estimation

Keywords
Adaptive Bayes procedure convergence rate minimax risk posterior distribution model selection

Citation

Belitser, Eduard; Ghosal, Subhashis. Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution. Ann. Statist. 31 (2003), no. 2, 536--559. doi:10.1214/aos/1051027880. https://projecteuclid.org/euclid.aos/1051027880


Export citation

References

  • BIRMAN, M. S. and SOLOMJAK, M. Z. (1967). Piecewise-poly nomial approximation of functions of the class W p. Soviet Math. Sb. 73 295-317.
  • BROWN, L. D. and LOW, M. G. (1996). Asy mptotic equivalence of nonparametric regression and white noise. Ann. Statist. 24 2384-2398.
  • BROWN, L. D., LOW, M. G. and ZHAO, L. H. (1997). Superefficiency in nonparametric function estimation. Ann. Statist. 25 2607-2625.
  • COX, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903-923.
  • DIACONIS, P. and FREEDMAN, D. (1997). On the Bernstein-von Mises theorem with infinite dimensional parameters. Technical Report 492, Dept. Statistics, Univ. California.
  • EFROMOVICH, S. Y. and PINSKER, M. S. (1984). Learning algorithm for nonparametric filtering. Autom. Remote Control 45 1434-1440.
  • FREEDMAN, D. (1999). On the Bernstein-von Mises theorem with infinite dimensional parameters. Ann. Statist. 27 1119-1140.
  • GHOSAL, S., GHOSH, J. K. and VAN DER VAART, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500-531.
  • GHOSAL, S., LEMBER, J. and VAN DER VAART, A. W. (2002). On Bayesian adaptation. In Proceedings of 8th Vilnius Conference on Probability and Statistics. To appear.
  • HUANG, T. M. (2000). Convergence rates for posterior distributions and adaptive estimation. Preprint.
  • KLEMELÄ, J. and NUSSBAUM, M. (1998). Constructive asy mptotic equivalence of density estimation and Gaussian white noise. Technical Report 53, Humboldt Univ., Berlin.
  • NUSSBAUM, M. (1996). Asy mptotic equivalence of density estimation and Gaussian white noise. Ann. Statist. 24 2399-2430.
  • PINSKER, M. S. (1980). Optimal filtration of square-integrable signals in Gaussian white noise. Problems Inform. Transmission 16 120-133.
  • SHEN, X. and WASSERMAN, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687-714.
  • VAN DER VAART, A. W. (2000). Private communication.
  • WILLIAMS, D. (1991). Probability with Martingales. Cambridge Univ. Press.
  • ZHAO, L. H. (2000). Bayesian aspects of some nonparametric problems. Ann. Statist. 28 532-552.
  • RALEIGH, NORTH CAROLINA 27695-8203 E-MAIL: ghosal@stat.ncsu.edu