Electronic Journal of Statistics

Posterior convergence and model estimation in Bayesian change-point problems

Heng Lian

Full-text: Open access

Abstract

We study the posterior distribution of the Bayesian multiple change-point regression problem when the number and the locations of the change-points are unknown. While it is relatively easy to apply the general theory to obtain the $O(1/\sqrt{n})$ rate up to some logarithmic factor, showing the parametric rate of convergence of the posterior distribution requires additional work and assumptions. Additionally, we demonstrate the asymptotic normality of the segment levels under these assumptions. For inferences on the number of change-points, we show that the Bayesian approach can produce a consistent posterior estimate. Finally, we show that consistent posterior for model selection necessarily implies that the parametric rate for posterior estimation stated previously cannot be uniform over the class of models we consider. This is the Bayesian version of the same phenomenon that has been noted and studied by other authors.

Article information

Source
Electron. J. Statist., Volume 4 (2010), 239-253.

Dates
First available in Project Euclid: 12 February 2010

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1265985088

Digital Object Identifier
doi:10.1214/09-EJS477

Mathematical Reviews number (MathSciNet)
MR2645483

Zentralblatt MATH identifier
1329.62219

Subjects
Primary: 62G20: Asymptotic properties
Secondary: 62G08: Nonparametric regression

Keywords
Change-point problems posterior distribution rate of convergence

Citation

Lian, Heng. Posterior convergence and model estimation in Bayesian change-point problems. Electron. J. Statist. 4 (2010), 239--253. doi:10.1214/09-EJS477. https://projecteuclid.org/euclid.ejs/1265985088


Export citation

References

  • [1] Barron, A., Schervish, M. J., and Wasserman, L. The consistency of posterior distributions in nonparametric problems., Annals of Statistics, 27(2):536–561, 1999.
  • [2] S. Ben Hariz, Wylie, J. J., and Zhang, Q. Optimal rate of convergence for nonparametric change-point estimators for nonstationary sequences., Annals of Statistics, 35(4) :1802–1826, 2007.
  • [3] Castillo, I. A semi-parametric Bernstein-von Mises theorem., Preprint, 2009.
  • [4] Choi, T. and Schervish, M. J. On posterior consistency in nonparametric regression problems., Journal of Multivariate Analysis, 98(10) :1969–1987, 2007.
  • [5] Fearnhead, P. Exact and efficient bayesian inference for multiple changepoint problems., Statistics and Computing, 16(2):203–213, 2006.
  • [6] Fearnhead, P. Computational methods for complex stochastic systems: a review of some alternatives to mcmc., Statistics and Computing, 18(2):151–171, 2008.
  • [7] Ghosal, S., Ghosh, J. K., and Ramamoorthi, R. V. Posterior consistency of Dirichlet mixtures in density estimation., Annals of Statistics, 27(1):143–158, 1999.
  • [8] Ghosal, S., Ghosh, J. K., and Samanta, T. On convergence of posterior distributions., Annals of Statistics, 23(6) :2145–2152, 1995.
  • [9] Ghosal, S., Ghosh, J. K., and Samanta, T. Approximation of the posterior distribution in a change-point problem., Annals of the Institute of Statistical Mathematics, 51(3):479–497, 1999.
  • [10] Ghosal, S., Ghosh, J. K., and Van der Vaart, A. W. Convergence rates of posterior distributions., Annals of Statistics, 28(2):500–531, 2000.
  • [11] Ghosal, S., Lember, J., and Van Der Vaart, A. Nonparametric Bayesian model selection and averaging., Electronic Journal of Statistics, 2:63–89, 2008.
  • [12] Ghosal, S. and Samanta, T. Asymptotic behaviour of Bayes estimates and posterior distribution in multiparameter nonregular cases., Mathematical Methods of Statistics, 4(4):361–388, 1995.
  • [13] Ghosal, S. and Van Der Vaart, A. Convergence rates of posterior distributions for noniid observations., Annals of Statistics, 35(1):192–223, 2007.
  • [14] Goldenshluger, A., Tsybakov, A., and Zeevi, A. Optimal change-point estimation from indirect observations., Annals of Statistics, 34(1):350–372, 2006.
  • [15] Green, P. J. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination., Biometrika, 82(4):711–732, 1995.
  • [16] Ibragimov, I. A. and Khasminskii, R. Z., Statistical estimation–asymptotic theory. Applications of mathematics. Springer-Verlag, New York, 1981.
  • [17] Ji, C. and Seymour, L. A consistent model selection procedure for Markov random fields based on penalized pseudolikelihood., Annals of Applied Probability, 6(2):423–443, 1996.
  • [18] Kim, Y. and Lee, J. A Bernstein-von Mises theorem in the nonparametric right-censoring model., Annals of Statistics, 32(4) :1492–1512, 2004.
  • [19] Lian, H. On the consistency of Bayesian function approximation using step functions., Neural Computation, 19(11) :2871–2880, 2007.
  • [20] Lian, H., Some topics on statistical theory and applications. PhD thesis, Brown University, 2007.
  • [21] Lian, H. Bayes and empirical Bayes inference in change-point problems., Communications in Statistics - Theory and Methods, 38(3):419–430, 2009.
  • [22] Lian, H., Thompson, W. A., Thurman, R., Stamatoyannopoulos, J. A., Noble, W. S., and Lawrence, C. E. Automated mapping of large-scale chromatin structure in encode., Bioinformatics, 24(17) :1911–1916, 2008.
  • [23] Liu, J. S. and Lawrence, C. E. Bayesian inference on biopolymer models., Bioinformatics, 15(1):38–52, 1999.
  • [24] Pflug, G. C. The limiting log-likelihood process for discontinuous density families., Zeitschrift Fur Wahrscheinlichkeitstheorie Und Verwandte Gebiete, 64(1):15–35, 1983.
  • [25] Polfeldt, T. Minimum variance order when estimating the location of an irregularity in the density., Annals of Mathematical Statistics, 41(2):673–679, 1970.
  • [26] Rivoirard, V. and Rousseau, J. Bernstein-von Mises theorem for linear functionals of the density., Preprint, 2009.
  • [27] Schwartz, L. On Bayes procedures., Z. Wahrsch. Verw. Gabiete, 4:10–26, 1965.
  • [28] Scricciolo, C. On rates of convergence for Bayesian density estimation., Scandinavian Journal of Statistics, 34(3):626–642, 2007.
  • [29] Shen, X. T. and Wasserman, L. Rates of convergence of posterior distributions., Annals of Statistics, 29(3):687–714, 2001.
  • [30] van der Vaart, A. W., Asymptotic statistics. Cambridge University Press, Cambridge, UK; New York, 1998.