Electronic Journal of Statistics

A novel approach to Bayesian consistency

Minwoo Chae and Stephen G. Walker

Full-text: Open access

Abstract

It is well-known that the Kullback–Leibler support condition implies posterior consistency in the weak topology, but is not sufficient for consistency in the total variation distance. There is a counter–example. Since then many authors have proposed sufficient conditions for strong consistency; and the aim of the present paper is to introduce new conditions with specific application to nonparametric mixture models with heavy–tailed components, such as the Student-$t$. The key is a more focused result on sets of densities where if strong consistency fails then it fails on such densities. This allows us to move away from the traditional types of sieves currently employed.

Article information

Source
Electron. J. Statist., Volume 11, Number 2 (2017), 4723-4745.

Dates
Received: May 2017
First available in Project Euclid: 24 November 2017

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1511492460

Digital Object Identifier
doi:10.1214/17-EJS1369

Mathematical Reviews number (MathSciNet)
MR3729657

Zentralblatt MATH identifier
06816631

Subjects
Primary: 62G05: Estimation 62G20: Asymptotic properties
Secondary: 62G07: Density estimation

Keywords
Kullback–Leibler divergence Lévy–Prokhorov metric mixture of Student’s $t$ distributions posterior consistency total variation

Rights
Creative Commons Attribution 4.0 International License.

Citation

Chae, Minwoo; Walker, Stephen G. A novel approach to Bayesian consistency. Electron. J. Statist. 11 (2017), no. 2, 4723--4745. doi:10.1214/17-EJS1369. https://projecteuclid.org/euclid.ejs/1511492460


Export citation

References

  • [1] Amewou-Atisso, M., Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. (2003). Posterior consistency for semi-parametric regression problems., Bernoulli 9 291–312.
  • [2] Andrews, D. F. and Mallows, C. L. (1974). Scale mixtures of normal distributions., Journal of the Royal Statistical Society: Series B (Statistical Methodology) 36 99–102.
  • [3] Barron, A., Schervish, M. J. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems., The Annals of Statistics 27 536–561.
  • [4] Canale, A. and De Blasi, P. (2017). Posterior consistency of nonparametric location-scale mixtures for multivariate density estimation., Bernoulli 23 379–404.
  • [5] Choi, T. and Schervish, M. J. (2007). On posterior consistency in nonparametric regression problems., Journal of Multivariate Analysis 98 1969–1987.
  • [6] De Blasi, P. and Walker, S. G. (2013). Bayesian asymptotics with misspecified models., Statistica Sinica 23 169–187.
  • [7] Diaconis, P. and Freedman, D. (1986a). On the consistency of Bayes estimates., The Annals of Statistics 14 1–26.
  • [8] Diaconis, P. and Freedman, D. (1986b). On inconsistent Bayes estimates of location., The Annals of Statistics 14 68–87.
  • [9] Doob, J. L. (1949). Application of the theory of martingales. In, Le Calcul des Probabilités et ses Applications 23–27. Colloques Internationaux du Centre National de la Recherche Scientifique Paris.
  • [10] Gerogiannis, D., Nikou, C. and Likas, A. (2009). The mixtures of Student’s t-distributions as a robust framework for rigid registration., Image and Vision Computing 27 1285–1294.
  • [11] Ghosal, S., Ghosh, J. K. and Ramamoorthi, R. (1999). Posterior consistency of Dirichlet mixtures in density estimation., The Annals of Statistics 27 143–158.
  • [12] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions., The Annals of Statistics 28 500–531.
  • [13] Ghosal, S. and Roy, A. (2006). Posterior consistency of Gaussian process prior for nonparametric binary regression., The Annals of Statistics 34 2413–2429.
  • [14] Ghosal, S. and van der Vaart, A. W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities., The Annals of Statistics 29 1233–1263.
  • [15] Ghosal, S. and van der Vaart, A. W. (2007). Posterior convergence rates of Dirichlet mixtures at smooth densities., The Annals of Statistics 35 697–723.
  • [16] Ghosal, S., Van Der Vaart, A. et al. (2007). Convergence rates of posterior distributions for noniid observations., The Annals of Statistics 35 192–223.
  • [17] Gibbs, A. L. and Su, F. E. (2002). On choosing and bounding probability metrics., International Statistical Review 70 419–435.
  • [18] Hejblum, B. P., Alkhassim, C., Gottardo, R., Caron, F. and Thiébaut, R. (2017). Sequential Dirichlet Process Mixtures of Multivariate Skew t-distributions for Model-based Clustering of Flow Cytometry Data., ArXiv:1702.04407.
  • [19] Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables., Journal of the American Statistical Association 58 13–30.
  • [20] Huber, P. J. (1981)., Robust statistics. John Wiley & Sons.
  • [21] Kleijn, B. J. and van der Vaart, A. W. (2006). Misspecification in infinite-dimensional Bayesian statistics., The Annals of Statistics 34 837–877.
  • [22] Kruijer, W., Rousseau, J. and Van Der Vaart, A. (2010). Adaptive Bayesian density estimation with location-scale mixtures., Electronic Journal of Statistics 4 1225–1257.
  • [23] Lijoi, A., Prünster, I. and Walker, S. G. (2005). On consistency of nonparametric normal mixtures for Bayesian density estimation., Journal of the American Statistical Association 100 1292–1296.
  • [24] Peel, D. and McLachlan, G. J. (2000). Robust mixture modelling using the t distribution., Statistics and Computing 10 339–348.
  • [25] Schwartz, L. (1965). On Bayes procedures., Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete 4 10–26.
  • [26] Shen, W., Tokdar, S. T. and Ghosal, S. (2013). Adaptive Bayesian multivariate density estimation with Dirichlet mixtures., Biometrika 100 623–640.
  • [27] Tokdar, S. T. (2006). Posterior consistency of Dirichlet location-scale mixture of normals in density estimation and regression., Sankhyā: The Indian Journal of Statistics 68 90–110.
  • [28] Walker, S. (2003). On sufficient conditions for Bayesian consistency., Biometrika 90 482–488.
  • [29] Walker, S. (2004). New approaches to Bayesian consistency., The Annals of Statistics 32 2028–2043.
  • [30] Walker, S. G., Lijoi, A. and Prünster, I. (2005). Data tracking and the understanding of Bayesian consistency., Biometrika 92 765–778.
  • [31] Walker, S. G., Lijoi, A. and Prünster, I. (2007). On rates of convergence for posterior distributions in infinite-dimensional models., The Annals of Statistics 35 738–746.
  • [32] Wong, W. H. and Shen, X. (1995). Probability inequalities for likelihood ratios and convergence rates of sieve MLEs., The Annals of Statistics 23 339–362.