The Annals of Statistics
- Ann. Statist.
- Volume 29, Issue 3 (2001), 687-714.
Rates of convergence of posterior distributions
Xiaotong Shen and Larry Wasserman
Full-text: Access has been disabled (more information)
Abstract
We compute the rate at which the posterior distribution concentrates around the true parameter value. The spaces we work in are quite general and include in finite dimensional cases. The rates are driven by two quantities: the size of the space, as measured by bracketing entropy, and the degree to which the prior concentrates in a small ball around the true parameter. We consider two examples.
Article information
Source
Ann. Statist. Volume 29, Issue 3 (2001), 687-714.
Dates
First available in Project Euclid: 24 December 2001
Permanent link to this document
http://projecteuclid.org/euclid.aos/1009210686
Digital Object Identifier
doi:10.1214/aos/1009210686
Mathematical Reviews number (MathSciNet)
MR1865337
Zentralblatt MATH identifier
1041.62022
Subjects
Primary: 62A15;secondary .
Secondary: 62E20: Asymptotic distribution theory 62G15: Tolerance and confidence regions
Keywords
Bayesian inference asymptotic inference non-parametric models sieves
Citation
Shen, Xiaotong; Wasserman, Larry. Rates of convergence of posterior distributions. Ann. Statist. 29 (2001), no. 3, 687--714. doi:10.1214/aos/1009210686. http://projecteuclid.org/euclid.aos/1009210686.
References
- Barron, A. (1988). The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions. Technical Report 7, Dept. Statistics, Univ. Illinois, Champaign.
- Barron, A., Schervish, M. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536-561. Mathematical Reviews (MathSciNet): MR2000k:62097
Zentralblatt MATH: 0980.62039
Digital Object Identifier: doi:10.1214/aos/1017939142
Project Euclid: euclid.aos/1018031206 - Barron, A. and Yang, Y. (1998). Anasymptotic property of model selectioncriteria. IEEE Trans. Inform. Theory 44 95-116.
- Barron, A., Birg´e, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301-413. Mathematical Reviews (MathSciNet): MR2000k:62049
Zentralblatt MATH: 0946.62036
Digital Object Identifier: doi:10.1007/s004400050210 - Berger, J. O. (1986). Discussion on the consistency of Bayes estimates (with discussion). Ann. Statist. 14 1-67. Mathematical Reviews (MathSciNet): MR829555
Digital Object Identifier: doi:10.1214/aos/1176349830
Project Euclid: euclid.aos/1176349830 - Birman, M. S. and Solomjak, M. Z. (1967). Piecewise-polynomial approximation of functions of the classes Wp. Mat. Sbornik 73 295-317. Mathematical Reviews (MathSciNet): MR36:576
- Brown, L. and Low, M. (1996). Asymptotic equivalence of non-parametric regression and white noise model. Ann. Statist. 24 2384-2398. Mathematical Reviews (MathSciNet): MR1425958
Zentralblatt MATH: 0867.62022
Digital Object Identifier: doi:10.1214/aos/1032181159
Project Euclid: euclid.aos/1032181159 - Cox, D. D. (1993). An analysis of Bayesian inference for non-parametric regression. Ann. Statist. 21 903-924. Mathematical Reviews (MathSciNet): MR1232525
Zentralblatt MATH: 0778.62003
Digital Object Identifier: doi:10.1214/aos/1176349157
Project Euclid: euclid.aos/1176349157 - Dembo, D., Mayer-Wolf, E. and Zeitouni, O. (1995). Exact behavior of Gaussianseminorms. Statist. Probab. Lett. 23 275-280.
- Dey, D., M ¨uller, P. and Sinha, D. (1998). Practical Nonparametric and Semiparametric Bayesian Statistics. Springer, New York.
- Diaconis, P. and Freedman, D. (1986). On the consistency of Bayes estimates (with discussion). Ann. Statist. 14 1-67.Mathematical Reviews (MathSciNet): MR88e:62016a
Digital Object Identifier: doi:10.1214/aos/1176349830
Project Euclid: euclid.aos/1176349830 - Diaconis, P. and Freedman, D. (1993). Non-parametric binary regression: A Bayesian approach. Ann. Statist. 21 2108-2137. Diaconis, P. and Freedman, D. (1997a). On the Bernstein-von Mises Theorem with infinite dimensional parameters. Unpublished manuscript. Diaconis, P. and Freedman, D. (1997b). Consistency of Bayes estimates for nonparametric regression: A review. In Festschrift for Lucien Le Cam (D. Pollard, E. Torgersenand G. Yang, eds.) Springer, New York. Mathematical Reviews (MathSciNet): MR1245784
Zentralblatt MATH: 0797.62031
Digital Object Identifier: doi:10.1214/aos/1176349413
Project Euclid: euclid.aos/1176349413 - Diaconis, P. and Freedman, D. (1998). Consistency of Bayes estimates for nonparametric regression: normal theory. Bernoulli 4 411-444. Mathematical Reviews (MathSciNet): MR2000b:62076
Digital Object Identifier: doi:10.2307/3318659
Project Euclid: euclid.bj/1173883814 - Doob, J. L. (1948). Applicationof the theory of martingales. Coll. Int. du C. N. R. S. Paris 22-28.
- Freedman, D. (1963). Onthe asymptotic behavior of Bayes estimates inthe discrete case I. Ann. Math. Statist. 34 1386-1403. Mathematical Reviews (MathSciNet): MR28:1706
Zentralblatt MATH: 0137.12603
Digital Object Identifier: doi:10.1214/aoms/1177703871
Project Euclid: euclid.aoms/1177703871 - Freedman, D. (1999). On the Bernstein-Von Mises theorem with infinite dimensional parameters. Ann. Statist. 27. Mathematical Reviews (MathSciNet): MR2001g:62005
Zentralblatt MATH: 0957.62002
Project Euclid: euclid.aos/1017938917 - Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1997). Non-informative priors via sieves and packing numbers. In Advances in Statistical Decision Theory and Applications (S. Panchapakesan and N. Balakrishnan, eds.) 119-132. Birkh¨auser, Boston. Ghosal, S., Ghosh, J. K. and Ramamoorthi, R.V. (1999a). Consistency issues in Bayesian nonparametrics. In Asymptotics, Nonparametrics and Time Series: A Tribute to Madan Lal Puri (S. Ghosh, ed.) 639-667. Dekker, New York Ghosal, S., Ghosh, J. K. and Ramamoorthi, R.V. (1999b). Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 143-158.
- Ghosal, S., Ghosh, J. K. and Van Der Vaart, A. (1998). Convergence rates of posterior distributions. Technical report, Free Univ. Amsterdam. Grenander U. (1981) Abstract Inference. Wiley, New York.
- Kuo, H. H. (1975). Gaussian Measures on Banach Spaces. Lecture Notes in Math. 463. Springer, Berlin.
- Shen, X. (1995). Onthe properties of Bayes procedures ingeneral parameter spaces. Technical report, Ohio State Univ.
- Shen, X. and Wong, W. H. (1994). Convergence rate of sieve estimates. Ann. Statist. 22 580-615. Zentralblatt MATH: 0805.62008
Mathematical Reviews (MathSciNet): MR1292531
Digital Object Identifier: doi:10.1214/aos/1176325486
Project Euclid: euclid.aos/1176325486 - Stone, C. (1982). Optimal global rates of convergence for non-parametric regression. Ann. Statist. 10 1040-1053. Mathematical Reviews (MathSciNet): MR673642
Zentralblatt MATH: 0511.62048
Digital Object Identifier: doi:10.1214/aos/1176345969
Project Euclid: euclid.aos/1176345969 - Schwartz, L. (1965). OnBayes' procedures. Z. Wahrsch. Verw. Gebiete 4 10-26.
- Triebel, H. (1983). Theory of Function Spaces. Birkh¨auser, Boston.
- van de Geer, S. (2000). Empirical Processes in M-Estimation. Cambridge Univ. Press.
- Wasserman, L. (1998). Asymptotic properties of nonparametric Bayesian procedures. In Practical Nonparametric and Semiparametric Bayesian Statistics (D. Dey, P. M ¨uller and D. Sinha, eds.) 293-304. Springer, New York.
- Wong, W. H. and Shen, X. (1995). A probability inequality for the likelihood surface and convergence rate of the maximum likelihood estimate. Ann. Statist. 23 339-362. Mathematical Reviews (MathSciNet): MR1332570
Zentralblatt MATH: 0829.62002
Digital Object Identifier: doi:10.1214/aos/1176324524
Project Euclid: euclid.aos/1176324524 - Zeidler, E. (1990). Nonlinear Functional Analysis and its Applications II/A. Springer, New York.
- Zhao, L. (1993). Frequentist and Bayesian aspects of some nonparametric estimation. Ph.D. dissertation, Cornell Univ.
- Zhao, L. (1998). A hierarchical Bayesian approach in nonparametric function estimation. Technical report, Dept. Statistics, Univ. Pennsylvania.
- You have access to this content.
- You have partial access to this content.
- You do not have access to this content.
More like this
- Posterior convergence rates of Dirichlet mixtures at smooth densities
Ghosal, Subhashis and van der Vaart, Aad, The Annals of Statistics, 2007 - Entropies and rates of convergence for maximum likelihood and
Bayes estimation for mixtures of normal densities
Ghosal, Subhashis and van der Vaart, Aad W., The Annals of Statistics, 2001 - Rates of contraction of posterior distributions based on Gaussian process priors
van der Vaart, A. W. and van Zanten, J. H., The Annals of Statistics, 2008
- Posterior convergence rates of Dirichlet mixtures at smooth densities
Ghosal, Subhashis and van der Vaart, Aad, The Annals of Statistics, 2007 - Entropies and rates of convergence for maximum likelihood and
Bayes estimation for mixtures of normal densities
Ghosal, Subhashis and van der Vaart, Aad W., The Annals of Statistics, 2001 - Rates of contraction of posterior distributions based on Gaussian process priors
van der Vaart, A. W. and van Zanten, J. H., The Annals of Statistics, 2008 - Misspecification in infinite-dimensional Bayesian statistics
Kleijn, B. J. K. and van der Vaart, A. W., The Annals of Statistics, 2006 - The consistency of posterior distributions in nonparametric
problems
Barron, Andrew, Schervish, Mark J., and Wasserman, Larry, The Annals of Statistics, 1999 - High-dimensional Bayesian inference in nonparametric additive models
Shang, Zuofeng and Li, Ping, Electronic Journal of Statistics, 2014 - Consistent estimation of joint distributions for sufficiently mixing random fields
Steif, Jeffrey E., The Annals of Statistics, 1997 - Bayesian inference with rescaled Gaussian process priors
van der Vaart, Aad and van Zanten, Harry, Electronic Journal of Statistics, 2007 - Convergence rates of posterior distributions for Brownian semimartingale models
Van Der Meulen, F.H., Van Der Vaart, Aad W., and Van Zanten, J.H., Bernoulli, 2006 - A Bernstein-Von Mises Theorem for discrete probability distributions
Boucheron, S. and Gassiat, E., Electronic Journal of Statistics, 2009