Remarks on consistency of posterior distributions
Taeryon Choi, R. V. Ramamoorthi
Abstract
In recent years, the literature in the area of Bayesian asymptotics has been rapidly growing. It is increasingly important to understand the concept of posterior consistency and validate specific Bayesian methods, in terms of consistency of posterior distributions. In this paper, we build up some conceptual issues in consistency of posterior distributions, and discuss panoramic views of them by comparing various approaches to posterior consistency that have been investigated in the literature. In addition, we provide interesting results on posterior consistency that deal with non-exponential consistency, improper priors and non i.i.d. (independent but not identically distributed) observations. We describe a few examples for illustrative purposes.
First Page:
Show
Hide
Primary Subjects: 62G20
Secondary Subjects: 62C10, 62F15
Keywords: affinity; Doob’s theorem; exponential consistency; posterior distribution; strongly separated
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.imsc/1209398468
Digital Object Identifier: doi:10.1214/074921708000000138
References
[1] Amewou-Atisso, M., Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (2003). Posterior consistency for semi-parametric regression problems. Bernoulli 9 291–312.
Mathematical Reviews (MathSciNet):
MR1997031
Digital Object Identifier: doi:10.3150/bj/1068128979
Project Euclid: euclid.bj/1068128979
[2] Barron, A. (1988). The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions. Technical Report 7, Dept. Statistics, Univ. Illinois, Champaign.
[3] Barron, A., Schervish, M. J. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536–561.
Mathematical Reviews (MathSciNet):
MR1714718
Zentralblatt MATH:
0980.62039
Digital Object Identifier: doi:10.1214/aos/1017939142
Project Euclid: euclid.aos/1018031206
[4] Birgé, L. (1983). Robust testing for independent non-identically distributed variables and Markov chains. In Practical Nonparametric and Semiparametric Bayesian Statistics. Lecture Notes in Statist. 16 (D. Dey, P. Müller and D. Sinha, eds.) 134–162. Springer, New York.
[5] Chatterji, S. D. and Mandrekar, V. (1978). Equivalence and Singularity of Gaussian Measures and Applications. Probabilistic Analysis and Related Topics 1. Academic Press, New York.
Mathematical Reviews (MathSciNet):
MR478320
Zentralblatt MATH:
0458.60028
[6] Choi, T. (2005). Posterior consistency in nonparametric regression problems under Gaussian process priors. Ph.D. thesis, Carnegie Mellon Univ., Pittsburgh, PA.
Mathematical Reviews (MathSciNet):
MR2707953
[7] Choi, T. (2007). Alternative posterior consistency results in nonparametric binary regression using gaussian process priors. J. Statist. Plann. Inference 137 2975–2983.
Mathematical Reviews (MathSciNet):
MR2323804
Zentralblatt MATH:
05182364
Digital Object Identifier: doi:10.1016/j.jspi.2006.11.001
[8] Choi, T. and Schervish, M. J. (2007). On posterior consistency in nonparametric regression problems. J. Multivariate Anal. 98 1969–1987.
Mathematical Reviews (MathSciNet):
MR2396949
Zentralblatt MATH:
1138.62020
Digital Object Identifier: doi:10.1016/j.jmva.2007.01.004
[9] Choudhuri, N., Ghosal, S. and Roy, A. (2004). Bayesian estimation of the spectral density of a time series. J. Amer. Statist. Assoc. 99 1050–1059.
Mathematical Reviews (MathSciNet):
MR2109494
Digital Object Identifier: doi:10.1198/016214504000000557
[10] Diaconis, P. and Freedman, D. A. (1986). On the consistency of Bayes estimates. Ann. Statist. 14 1–26.
Mathematical Reviews (MathSciNet):
MR829555
Digital Object Identifier: doi:10.1214/aos/1176349830
Project Euclid: euclid.aos/1176349830
[11] Doob, J. L. (1949). Application of the theory of martingales. Coll. Int. du C. N. R. S. Paris. 23–27.
Mathematical Reviews (MathSciNet):
MR33460
Zentralblatt MATH:
0041.45101
[12] Freedman, D. A. (1963). On the asymptotic behavior of Bayes’ estimates in the discrete case. Ann. Math. Statist. 34 1386–1403.
Mathematical Reviews (MathSciNet):
MR158483
Zentralblatt MATH:
0137.12603
Digital Object Identifier: doi:10.1214/aoms/1177703871
Project Euclid: euclid.aoms/1177703871
[13] Freedman, D. A. (1965). On the asymptotic behavior of Bayes’ estimates in the discrete case. II. Ann. Math. Statist. 36 454–456.
Mathematical Reviews (MathSciNet):
MR174146
Digital Object Identifier: doi:10.1214/aoms/1177700155
Project Euclid: euclid.aoms/1177700155
[14] Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999a). Consistency issues in Bayesian nonparametrics. In Asymptotics, Nonparametrics, and Time Series. Statist. Textbooks Monogr. 158 639–667. Dekker, New York.
Mathematical Reviews (MathSciNet):
MR1724711
Zentralblatt MATH:
1069.62516
[15] Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. V. (1999b). Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 143–158.
Mathematical Reviews (MathSciNet):
MR1701105
Zentralblatt MATH:
1055.62100
Digital Object Identifier: doi:10.1214/aos/1018031105
Project Euclid: euclid.aos/1018031105
[16] Ghosal, S. and Roy, A. (2006). Posterior consistency of Gaussian process prior for nonparametric binary regression. Ann. Statist. 34 2413–2429.
Mathematical Reviews (MathSciNet):
MR2291505
Zentralblatt MATH:
1106.62039
Digital Object Identifier: doi:10.1214/009053606000000795
Project Euclid: euclid.aos/1169571802
[17] Ghosal, S. and van der Vaart, A. W. (2007). Convergence rates of posterior distributions for noniid observations. Ann. Statist. 35 192–223.
[18] Ghosh, J. K., Delampady, M. and Samanta, T. (2006). An Introduction to Bayesian Analysis Theory and Methods. Springer, New York.
Mathematical Reviews (MathSciNet):
MR2247439
Zentralblatt MATH:
1135.62002
[19] Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics. Springer, New York.
Mathematical Reviews (MathSciNet):
MR1992245
[20] Kass, R. E. and Wasserman, L. (1996). The selection of prior distributions by formal rules. J. Amer. Statist. Assoc. 91 1343–1370.
[21] Kraft, C. (1955). Some conditions for consistency and uniform consistency of statistical procedures. Univ. California Publ. Statist. 2 125–141.
Mathematical Reviews (MathSciNet):
MR73896
[22] Le Cam, L. M. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York.
Mathematical Reviews (MathSciNet):
MR856411
Zentralblatt MATH:
0605.62002
[23] Li, W. and Shao, Q.-M. (2001). Gaussian processes: inequalities, small ball probabilities and applications. In Stochastic Processes: Theory and Methods. Handbook of Statist. 19 533–597. North-Holland, Amsterdam.
Mathematical Reviews (MathSciNet):
MR1861734
Zentralblatt MATH:
0987.60053
[24] Salinetti, G. (2003). New tools for consistency in Bayesian nonparametrics. In Bayesian Statistics 7 369–384. Oxford Univ. Press, New York.
Mathematical Reviews (MathSciNet):
MR2003184
[25] Schwartz, L. (1965). On Bayes procedures. Z. Wahrsch. Verw. Gebiete 4 10–26.
Mathematical Reviews (MathSciNet):
MR184378
Digital Object Identifier: doi:10.1007/BF00535479
[26] Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 666–686.
Mathematical Reviews (MathSciNet):
MR1865337
Zentralblatt MATH:
1041.62022
Digital Object Identifier: doi:10.1214/aos/1009210686
Project Euclid: euclid.aos/1009210686
[27] Shepp, L. A. (1965). Distingunishing a sequence of random variables from a translate of itself. Ann. Math. Statist. 36 1107–1112.
Mathematical Reviews (MathSciNet):
MR176509
Digital Object Identifier: doi:10.1214/aoms/1177699985
Project Euclid: euclid.aoms/1177699985
[28] Srivastava, S. M. (1998). A Course on Borel Sets. Springer, Berlin.
Mathematical Reviews (MathSciNet):
MR1619545
Zentralblatt MATH:
0903.28001
[29] Tokdar, S. and Ghosh, J. K. (2007). Posterior consistency of Gaussian process priors in density estimation. J. Statist. Plann. Inference 137 34–42.
Mathematical Reviews (MathSciNet):
MR2292838
Zentralblatt MATH:
1098.62041
Digital Object Identifier: doi:10.1016/j.jspi.2005.09.005
[30] van de Geer, S. (1993). Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Statist. 21 14–44.
Mathematical Reviews (MathSciNet):
MR1212164
Zentralblatt MATH:
0779.62033
Digital Object Identifier: doi:10.1214/aos/1176349013
Project Euclid: euclid.aos/1176349013
[31] van der Vaart, A. W. and van Zanten, J. (2007). Rates of contraction of posterior distributions based on Gaussian process priors. Ann. Statist. To appear.
Mathematical Reviews (MathSciNet):
MR2418663
Zentralblatt MATH:
1141.60018
Digital Object Identifier: doi:10.1214/009053607000000613
Project Euclid: euclid.aos/1211819570
[32] Walker, S. (2004). New approaches to Bayesian consistency. Ann. Statist. 32 2028–2043.
Mathematical Reviews (MathSciNet):
MR2102501
Zentralblatt MATH:
1056.62040
Digital Object Identifier: doi:10.1214/009053604000000409
Project Euclid: euclid.aos/1098883780
[33] Walker, S. G. (2003). Bayesian consistency for a class of regression problems. South African Stat. J. 37 149–167.
Mathematical Reviews (MathSciNet):
MR2042627
Zentralblatt MATH:
1065.62038
Institute of Mathematical Statistics Collections
