We study the posterior distribution of the Bayesian multiple change-point regression problem when the number and the locations of the change-points are unknown. While it is relatively easy to apply the general theory to obtain the
rate up to some logarithmic factor, showing the parametric rate of convergence of the posterior distribution requires additional work and assumptions. Additionally, we demonstrate the asymptotic normality of the segment levels under these assumptions. For inferences on the number of change-points, we show that the Bayesian approach can produce a consistent posterior estimate. Finally, we show that consistent posterior for model selection necessarily implies that the parametric rate for posterior estimation stated previously cannot be uniform over the class of models we consider. This is the Bayesian version of the same phenomenon that has been noted and studied by other authors.
References
[1] Barron, A., Schervish, M. J., and Wasserman, L. The consistency of posterior distributions in nonparametric problems., Annals of Statistics, 27(2):536–561, 1999.
[2] S. Ben Hariz, Wylie, J. J., and Zhang, Q. Optimal rate of convergence for nonparametric change-point estimators for nonstationary sequences., Annals of Statistics, 35(4) :1802–1826, 2007.
[3] Castillo, I. A semi-parametric Bernstein-von Mises theorem., Preprint, 2009.
[4] Choi, T. and Schervish, M. J. On posterior consistency in nonparametric regression problems., Journal of Multivariate Analysis, 98(10) :1969–1987, 2007.
[5] Fearnhead, P. Exact and efficient bayesian inference for multiple changepoint problems., Statistics and Computing, 16(2):203–213, 2006.
[6] Fearnhead, P. Computational methods for complex stochastic systems: a review of some alternatives to mcmc., Statistics and Computing, 18(2):151–171, 2008.
[7] Ghosal, S., Ghosh, J. K., and Ramamoorthi, R. V. Posterior consistency of Dirichlet mixtures in density estimation., Annals of Statistics, 27(1):143–158, 1999.
[8] Ghosal, S., Ghosh, J. K., and Samanta, T. On convergence of posterior distributions., Annals of Statistics, 23(6) :2145–2152, 1995.
[9] Ghosal, S., Ghosh, J. K., and Samanta, T. Approximation of the posterior distribution in a change-point problem., Annals of the Institute of Statistical Mathematics, 51(3):479–497, 1999.
[10] Ghosal, S., Ghosh, J. K., and Van der Vaart, A. W. Convergence rates of posterior distributions., Annals of Statistics, 28(2):500–531, 2000.
[11] Ghosal, S., Lember, J., and Van Der Vaart, A. Nonparametric Bayesian model selection and averaging., Electronic Journal of Statistics, 2:63–89, 2008.
[12] Ghosal, S. and Samanta, T. Asymptotic behaviour of Bayes estimates and posterior distribution in multiparameter nonregular cases., Mathematical Methods of Statistics, 4(4):361–388, 1995.
[13] Ghosal, S. and Van Der Vaart, A. Convergence rates of posterior distributions for noniid observations., Annals of Statistics, 35(1):192–223, 2007.
[14] Goldenshluger, A., Tsybakov, A., and Zeevi, A. Optimal change-point estimation from indirect observations., Annals of Statistics, 34(1):350–372, 2006.
[15] Green, P. J. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination., Biometrika, 82(4):711–732, 1995.
[16] Ibragimov, I. A. and Khasminskii, R. Z., Statistical estimation–asymptotic theory. Applications of mathematics. Springer-Verlag, New York, 1981.
Mathematical Reviews (MathSciNet):
MR620321
[17] Ji, C. and Seymour, L. A consistent model selection procedure for Markov random fields based on penalized pseudolikelihood., Annals of Applied Probability, 6(2):423–443, 1996.
[18] Kim, Y. and Lee, J. A Bernstein-von Mises theorem in the nonparametric right-censoring model., Annals of Statistics, 32(4) :1492–1512, 2004.
[19] Lian, H. On the consistency of Bayesian function approximation using step functions., Neural Computation, 19(11) :2871–2880, 2007.
[20] Lian, H., Some topics on statistical theory and applications. PhD thesis, Brown University, 2007.
[21] Lian, H. Bayes and empirical Bayes inference in change-point problems., Communications in Statistics - Theory and Methods, 38(3):419–430, 2009.
[22] Lian, H., Thompson, W. A., Thurman, R., Stamatoyannopoulos, J. A., Noble, W. S., and Lawrence, C. E. Automated mapping of large-scale chromatin structure in encode., Bioinformatics, 24(17) :1911–1916, 2008.
[23] Liu, J. S. and Lawrence, C. E. Bayesian inference on biopolymer models., Bioinformatics, 15(1):38–52, 1999.
[24] Pflug, G. C. The limiting log-likelihood process for discontinuous density families., Zeitschrift Fur Wahrscheinlichkeitstheorie Und Verwandte Gebiete, 64(1):15–35, 1983.
Mathematical Reviews (MathSciNet):
MR710646
[25] Polfeldt, T. Minimum variance order when estimating the location of an irregularity in the density., Annals of Mathematical Statistics, 41(2):673–679, 1970.
Mathematical Reviews (MathSciNet):
MR256500
[26] Rivoirard, V. and Rousseau, J. Bernstein-von Mises theorem for linear functionals of the density., Preprint, 2009.
[27] Schwartz, L. On Bayes procedures., Z. Wahrsch. Verw. Gabiete, 4:10–26, 1965.
Mathematical Reviews (MathSciNet):
MR184378
[28] Scricciolo, C. On rates of convergence for Bayesian density estimation., Scandinavian Journal of Statistics, 34(3):626–642, 2007.
[29] Shen, X. T. and Wasserman, L. Rates of convergence of posterior distributions., Annals of Statistics, 29(3):687–714, 2001.
[30] van der Vaart, A. W., Asymptotic statistics. Cambridge University Press, Cambridge, UK; New York, 1998.