Electronic Journal of Statistics

Posterior convergence and model estimation in Bayesian change-point problems

Heng Lian
Source: Electron. J. Statist. Volume 4 (2010), 239-253.

Abstract

We study the posterior distribution of the Bayesian multiple change-point regression problem when the number and the locations of the change-points are unknown. While it is relatively easy to apply the general theory to obtain the $O(1/\sqrt{n})$ rate up to some logarithmic factor, showing the parametric rate of convergence of the posterior distribution requires additional work and assumptions. Additionally, we demonstrate the asymptotic normality of the segment levels under these assumptions. For inferences on the number of change-points, we show that the Bayesian approach can produce a consistent posterior estimate. Finally, we show that consistent posterior for model selection necessarily implies that the parametric rate for posterior estimation stated previously cannot be uniform over the class of models we consider. This is the Bayesian version of the same phenomenon that has been noted and studied by other authors.

First Page: Show Hide
Primary Subjects: 62G20
Secondary Subjects: 62G08
Full-text: Open access
Links and Identifiers

Permanent link to this document: http://projecteuclid.org/euclid.ejs/1265985088
Digital Object Identifier: doi:10.1214/09-EJS477
Mathematical Reviews number (MathSciNet): MR2645483

References

[1] Barron, A., Schervish, M. J., and Wasserman, L. The consistency of posterior distributions in nonparametric problems., Annals of Statistics, 27(2):536–561, 1999.
Mathematical Reviews (MathSciNet): MR1714718
Zentralblatt MATH: 0980.62039
Digital Object Identifier: doi:10.1214/aos/1017939142
Project Euclid: euclid.aos/1018031206
[2] S. Ben Hariz, Wylie, J. J., and Zhang, Q. Optimal rate of convergence for nonparametric change-point estimators for nonstationary sequences., Annals of Statistics, 35(4) :1802–1826, 2007.
Mathematical Reviews (MathSciNet): MR2351106
Zentralblatt MATH: 1147.62043
Digital Object Identifier: doi:10.1214/009053606000001596
Project Euclid: euclid.aos/1188405631
[3] Castillo, I. A semi-parametric Bernstein-von Mises theorem., Preprint, 2009.
[4] Choi, T. and Schervish, M. J. On posterior consistency in nonparametric regression problems., Journal of Multivariate Analysis, 98(10) :1969–1987, 2007.
Mathematical Reviews (MathSciNet): MR2396949
Zentralblatt MATH: 1138.62020
Digital Object Identifier: doi:10.1016/j.jmva.2007.01.004
[5] Fearnhead, P. Exact and efficient bayesian inference for multiple changepoint problems., Statistics and Computing, 16(2):203–213, 2006.
Mathematical Reviews (MathSciNet): MR2227396
Digital Object Identifier: doi:10.1007/s11222-006-8450-8
[6] Fearnhead, P. Computational methods for complex stochastic systems: a review of some alternatives to mcmc., Statistics and Computing, 18(2):151–171, 2008.
Mathematical Reviews (MathSciNet): MR2390816
Digital Object Identifier: doi:10.1007/s11222-007-9045-8
[7] Ghosal, S., Ghosh, J. K., and Ramamoorthi, R. V. Posterior consistency of Dirichlet mixtures in density estimation., Annals of Statistics, 27(1):143–158, 1999.
Mathematical Reviews (MathSciNet): MR1701105
Zentralblatt MATH: 0932.62043
Digital Object Identifier: doi:10.1214/aos/1018031105
Project Euclid: euclid.aos/1018031105
[8] Ghosal, S., Ghosh, J. K., and Samanta, T. On convergence of posterior distributions., Annals of Statistics, 23(6) :2145–2152, 1995.
Mathematical Reviews (MathSciNet): MR1389869
Zentralblatt MATH: 0858.62024
Digital Object Identifier: doi:10.1214/aos/1034713651
Project Euclid: euclid.aos/1034713651
[9] Ghosal, S., Ghosh, J. K., and Samanta, T. Approximation of the posterior distribution in a change-point problem., Annals of the Institute of Statistical Mathematics, 51(3):479–497, 1999.
Mathematical Reviews (MathSciNet): MR1722841
Zentralblatt MATH: 0938.62023
Digital Object Identifier: doi:10.1023/A:1003998005295
[10] Ghosal, S., Ghosh, J. K., and Van der Vaart, A. W. Convergence rates of posterior distributions., Annals of Statistics, 28(2):500–531, 2000.
Mathematical Reviews (MathSciNet): MR1790007
Digital Object Identifier: doi:10.1214/aos/1016218228
Project Euclid: euclid.aos/1016218228
[11] Ghosal, S., Lember, J., and Van Der Vaart, A. Nonparametric Bayesian model selection and averaging., Electronic Journal of Statistics, 2:63–89, 2008.
Mathematical Reviews (MathSciNet): MR2386086
Zentralblatt MATH: 1135.62028
Digital Object Identifier: doi:10.1214/07-EJS090
Project Euclid: euclid.ejs/1201877208
[12] Ghosal, S. and Samanta, T. Asymptotic behaviour of Bayes estimates and posterior distribution in multiparameter nonregular cases., Mathematical Methods of Statistics, 4(4):361–388, 1995.
Mathematical Reviews (MathSciNet): MR1372011
Zentralblatt MATH: 0840.62033
[13] Ghosal, S. and Van Der Vaart, A. Convergence rates of posterior distributions for noniid observations., Annals of Statistics, 35(1):192–223, 2007.
[14] Goldenshluger, A., Tsybakov, A., and Zeevi, A. Optimal change-point estimation from indirect observations., Annals of Statistics, 34(1):350–372, 2006.
Mathematical Reviews (MathSciNet): MR2275245
Zentralblatt MATH: 1091.62021
Digital Object Identifier: doi:10.1214/009053605000000750
Project Euclid: euclid.aos/1146576266
[15] Green, P. J. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination., Biometrika, 82(4):711–732, 1995.
Mathematical Reviews (MathSciNet): MR1380810
Zentralblatt MATH: 0861.62023
Digital Object Identifier: doi:10.1093/biomet/82.4.711
[16] Ibragimov, I. A. and Khasminskii, R. Z., Statistical estimation–asymptotic theory. Applications of mathematics. Springer-Verlag, New York, 1981.
Mathematical Reviews (MathSciNet): MR620321
[17] Ji, C. and Seymour, L. A consistent model selection procedure for Markov random fields based on penalized pseudolikelihood., Annals of Applied Probability, 6(2):423–443, 1996.
Mathematical Reviews (MathSciNet): MR1398052
Zentralblatt MATH: 0856.62082
Digital Object Identifier: doi:10.1214/aoap/1034968138
Project Euclid: euclid.aoap/1034968138
[18] Kim, Y. and Lee, J. A Bernstein-von Mises theorem in the nonparametric right-censoring model., Annals of Statistics, 32(4) :1492–1512, 2004.
Mathematical Reviews (MathSciNet): MR2089131
Zentralblatt MATH: 1047.62043
Digital Object Identifier: doi:10.1214/009053604000000175
Project Euclid: euclid.aos/1091626176
[19] Lian, H. On the consistency of Bayesian function approximation using step functions., Neural Computation, 19(11) :2871–2880, 2007.
Mathematical Reviews (MathSciNet): MR2352968
Zentralblatt MATH: 1144.62310
Digital Object Identifier: doi:10.1162/neco.2007.19.11.2871
[20] Lian, H., Some topics on statistical theory and applications. PhD thesis, Brown University, 2007.
Mathematical Reviews (MathSciNet): MR2710471
[21] Lian, H. Bayes and empirical Bayes inference in change-point problems., Communications in Statistics - Theory and Methods, 38(3):419–430, 2009.
Mathematical Reviews (MathSciNet): MR2510794
Zentralblatt MATH: 1159.62014
Digital Object Identifier: doi:10.1080/03610920802220801
[22] Lian, H., Thompson, W. A., Thurman, R., Stamatoyannopoulos, J. A., Noble, W. S., and Lawrence, C. E. Automated mapping of large-scale chromatin structure in encode., Bioinformatics, 24(17) :1911–1916, 2008.
[23] Liu, J. S. and Lawrence, C. E. Bayesian inference on biopolymer models., Bioinformatics, 15(1):38–52, 1999.
[24] Pflug, G. C. The limiting log-likelihood process for discontinuous density families., Zeitschrift Fur Wahrscheinlichkeitstheorie Und Verwandte Gebiete, 64(1):15–35, 1983.
Mathematical Reviews (MathSciNet): MR710646
[25] Polfeldt, T. Minimum variance order when estimating the location of an irregularity in the density., Annals of Mathematical Statistics, 41(2):673–679, 1970.
Mathematical Reviews (MathSciNet): MR256500
Zentralblatt MATH: 0198.23402
Digital Object Identifier: doi:10.1214/aoms/1177697112
Project Euclid: euclid.aoms/1177697112
[26] Rivoirard, V. and Rousseau, J. Bernstein-von Mises theorem for linear functionals of the density., Preprint, 2009.
[27] Schwartz, L. On Bayes procedures., Z. Wahrsch. Verw. Gabiete, 4:10–26, 1965.
Mathematical Reviews (MathSciNet): MR184378
Digital Object Identifier: doi:10.1007/BF00535479
[28] Scricciolo, C. On rates of convergence for Bayesian density estimation., Scandinavian Journal of Statistics, 34(3):626–642, 2007.
Mathematical Reviews (MathSciNet): MR2368802
Digital Object Identifier: doi:10.1111/j.1467-9469.2006.00540.x
[29] Shen, X. T. and Wasserman, L. Rates of convergence of posterior distributions., Annals of Statistics, 29(3):687–714, 2001.
Mathematical Reviews (MathSciNet): MR1865337
Zentralblatt MATH: 1041.62022
Digital Object Identifier: doi:10.1214/aos/1009210686
Project Euclid: euclid.aos/1009210686
[30] van der Vaart, A. W., Asymptotic statistics. Cambridge University Press, Cambridge, UK; New York, 1998.
Mathematical Reviews (MathSciNet): MR1652247
Zentralblatt MATH: 0910.62001

2012 © Institute of Mathematical Statistics

Electronic Journal of Statistics

Electronic Journal of Statistics