Electronic Journal of Statistics

Efficient semiparametric estimation and model selection for multidimensional mixtures

Elisabeth Gassiat, Judith Rousseau, and Elodie Vernet

Full-text: Open access

Abstract

In this paper, we consider nonparametric multidimensional finite mixture models and we are interested in the semiparametric estimation of the population weights. Here, the i.i.d. observations are assumed to have at least three components which are independent given the population. We approximate the semiparametric model by projecting the conditional distributions on step functions associated to some partition. Our first main result is that if we refine the partition slowly enough, the associated sequence of maximum likelihood estimators of the weights is asymptotically efficient, and the posterior distribution of the weights, when using a Bayesian procedure, satisfies a semiparametric Bernstein-von Mises theorem. We then propose a cross-validation like method to select the partition in a finite horizon. Our second main result is that the proposed procedure satisfies an oracle inequality. Numerical experiments on simulated data illustrate our theoretical results.

Article information

Source
Electron. J. Statist., Volume 12, Number 1 (2018), 703-740.

Dates
Received: September 2016
First available in Project Euclid: 27 February 2018

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1519700499

Digital Object Identifier
doi:10.1214/17-EJS1387

Mathematical Reviews number (MathSciNet)
MR3769193

Zentralblatt MATH identifier
06864474

Subjects
Primary: 62G05: Estimation 62G20: Asymptotic properties

Keywords
Semiparametric statistics mixture models efficiency Bernstein von Mises theorem

Rights
Creative Commons Attribution 4.0 International License.

Citation

Gassiat, Elisabeth; Rousseau, Judith; Vernet, Elodie. Efficient semiparametric estimation and model selection for multidimensional mixtures. Electron. J. Statist. 12 (2018), no. 1, 703--740. doi:10.1214/17-EJS1387. https://projecteuclid.org/euclid.ejs/1519700499


Export citation

References

  • [1] E. S. Allman, C. Matias, and J. A. Rhodes. Identifiability of parameters in latent structure models with many observed variables., Ann. Statist., 37(6A) :3099–3132, 12 2009.
  • [2] A. Anandkumar, R. Ge, D. Hsu, S. M. Kakade, and M. Telgarsky. Tensor decompositions for learning latent variable models., JMLR, 15 :2773–2832, 2014.
  • [3] T. Ando., Bayesian model selection and statistical modeling. Statistics: Textbooks and Monographs. CRC Press, Boca Raton, FL, 2010.
  • [4] S. Arlot., Contributions to statistical learning theory: estimator selection and change-point detection. Habilitation à diriger des recherches, University Paris Diderot, December 2014. Habilitation à diriger des recherches.
  • [5] S. Arlot and A. Celisse. A survey of cross-validation procedures for model selection., Stat. Surv., 4:40–79, 2010.
  • [6] P. Barbe and P. Bertail., The Weighted Bootstrap, volume 98 of Lecture Notes in Statistics. Springer, 1995.
  • [7] J.-P. Baudry, C. Maugis, and B. Michel. Slope heuristics: overview and implementation., Stat. Comput., (22):455–470, 2012.
  • [8] P. J. Bickel, C. A. J. Klaassen, Y. Ritov, and J. A. Wellner., Efficient and adaptive estimation for semiparametric models. Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins University Press, Baltimore, MD, 1993.
  • [9] P. J. Bickel and B. J. K. Kleijn. The semiparametric Bernstein-von Mises theorem., Ann. Statist., 40(1):206–237, 2012.
  • [10] S. Bonhomme, K. Jochmans, and J.-M. Robin. Estimating multivariate latent-structure models., Ann. Statist., 44(2):540–563, 2016.
  • [11] S. Bonhomme, K. Jochmans, and J.-M. Robin. Non-parametric estimation of finite mixtures from repeated measurements., J. R. Stat. Soc. Ser. B. Stat. Methodol., 78(1):211–229, 2016.
  • [12] S. Boucheron and E. Gassiat. A Bernstein-von Mises theorem for discrete probability distributions., Electron. J. Stat., 3:114–148, 2009.
  • [13] M. A. Brookhart and M. J. van der Laan. A semiparametric model selection criterion with applications to the marginal structural model., Comput. Statist. Data Anal., 50(2):475–498, 2006.
  • [14] I. Castillo. Semiparametric Bernstein–von Mises theorem and bias, illustrated with Gaussian process priors., Sankhya A, 74(2):194–221, 2012.
  • [15] I. Castillo. A semiparametric Bernstein–von Mises theorem for Gaussian process priors., Probab. Theory Related Fields, 152(1–2):53–99, 2012.
  • [16] G. Claeskens and N. L. Hjort., Model selection and model averaging, volume 27 of Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, 2008.
  • [17] P. De Blasi and N. L. Hjort. The Bernstein–von Mises theorem in semiparametric competing risks models., J. Statist. Plann. Inference, 139(7) :2316–2328, 2009.
  • [18] Y. De Castro, E. Gassiat, and C. Lacour. Minimax adaptive estimation of nonparametric hidden Markov models., JMLR, 17(111), 2016.
  • [19] Y. De Castro, E. Gassiat, and S. Le Corff. Consistent estimation of the filtering and marginal smoothing distributions in nonparametric hidden Markov models., I.E.E.E. Trans. Info. Th., 63(8) :4758–4777, 2017.
  • [20] E. Gassiat, D. Pollard, and G. Stoltz. Revisiting the van Trees inequality in the spirit of Hajek and Le Cam., unpublished manuscript, 2013.
  • [21] E. Gassiat and J. Rousseau. Non parametric finite translation hidden Markov models and extensions., Bernoulli, 22(1):193–212, 2016.
  • [22] M. H. Hansen and B. Yu. Model selection and the principle of minimum description length. 96:746–774, 2001.
  • [23] J. B. Kruskal. Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics., Linear Algebra and Appl., 18(2):95–138, 1977.
  • [24] L. Le Cam and G. Yang., Asymptotics in Statistics. Some Basic Concepts, Second Edition. Springer-Verlag, New-York, 2000.
  • [25] G. Lee and C. Scott. EM algorithms for multivariate Gaussian mixture models with truncated and censored data., Comput. Statist. Data Anal., 56(9) :2816–2829, 2012.
  • [26] P. Massart., Concentration inequalities and model selection, volume 1896 of Lecture Notes in Mathematics. Springer, Berlin, 2007. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6–23, 2003, With a foreword by Jean Picard.
  • [27] B. McNeney and J. A. Wellner. Application of convolution theorems in semiparametric models with non-i.i.d. data., J. Statist. Plann. Inference, 91(2):441–480, 2000. Prague Workshop on Perspectives in Modern Statistical Inference: Parametrics, Semi-parametrics, Non-parametrics (1998).
  • [28] J. A. Rhodes. A concise proof of kruskal’s theorem on tensor decomposition., Linear Algebra and Appl., 432(7) :1818–1824, 2010.
  • [29] V. Rivoirard and J. Rousseau. Bernstein-von Mises theorem for linear functionals of the density., Ann. Statist., 40(3) :1489–1523, 2012.
  • [30] C. P. Robert., The Bayesian Choice. Springer-Verlag, New York, second edition, 2001.
  • [31] X. Shen. Asymptotic normality of semiparametric and nonparametric posterior distributions., J. Amer. Statist. Assoc., 97(457):222–235, 2002.
  • [32] E. M. Stein and R. Shakarchi., Real analysis. Princeton Lectures in Analysis, III. Princeton University Press, Princeton, NJ, 2005. Measure theory, integration, and Hilbert spaces.
  • [33] A. W. van der Vaart., Asymptotic statistics, volume 3 of Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, 1998.
  • [34] A. W. van der Vaart. Semiparametric statistics. In, Lectures on probability theory and statistics (Saint-Flour, 1999), volume 1781 of Lecture Notes in Math., pages 331–457. Springer, Berlin, 2002.
  • [35] E. Vernet. Posterior consistency for nonparametric Hidden Markov Models with finite state space., Electronic Journal of Statistics, 9:717–752, 2015.