Electronic Journal of Statistics

Mixture of Gaussian regressions model with logistic weights, a penalized maximum likelihood approach

L. Montuelle and E. Le Pennec

Full-text: Open access


In the framework of conditional density estimation, we use candidates taking the form of mixtures of Gaussian regressions with logistic weights and means depending on the covariate. We aim at estimating the number of components of this mixture, as well as the other parameters, by a penalized maximum likelihood approach. We provide a lower bound on the penalty that ensures an oracle inequality for our estimator. We perform some numerical experiments that support our theoretical analysis.

Article information

Electron. J. Statist. Volume 8, Number 1 (2014), 1661-1695.

First available in Project Euclid: 11 September 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression

Mixture of Gaussian regressions models mixture of regressions models penalized likelihood model selection


Montuelle, L.; Le Pennec, E. Mixture of Gaussian regressions model with logistic weights, a penalized maximum likelihood approach. Electron. J. Statist. 8 (2014), no. 1, 1661--1695. doi:10.1214/14-EJS939. https://projecteuclid.org/euclid.ejs/1410441420

Export citation


  • Antoniadis, A., Bigot, J., and von Sachs, R., A multiscale approach for statistical characterization of functional images. Journal of Computational and Graphical Statistics, 18, 2009.
  • Barron, A., Huang, C., Li, J., and Luo, X., The mdl principle, penalized likelihoods, and statistical risk. Festschrift for Jorma Rissanen. Tampere University Press, Tampere, Finland, 2008.
  • Baudry, J.-P., Maugis, C., and Michel, B., Slope heuristics: Overview and implementation. Statistics and Computing, 22, 2011.
  • Biernacki, Ch. and Castellan, G., A data-driven bound on variances for avoiding degeneracy in univariate gaussian mixtures. Pub IRMA Lille, 71, 2011.
  • Birgé, L. and Massart, P., Minimal penalties for gaussian model selection. Probability Theory and Related Fields, 138(1–2):33–73, 2007. ISSN 0178-8051. 10.1007/s00440-006-0011-8.
  • Brinkman, N. D., Ethanol fuel-a single-cylinder engine study of efficiency and exhaust emissions. SAE Technical Paper, 810345, 1981.
  • Burnham, K. P. and Anderson, D. R., Model Selection and Multimodel Inference. A Practical Information-Theoretic Approach. Springer-Verlag, New-York, 2nd edition, 2002.
  • Celeux, G. and Govaert, G., Gaussian parsimonious clustering models. Pattern Recognition, 28(5), 1995.
  • Chamroukhi, F., Samé, A., Govaert, G., and Aknin, P., A hidden process regression model for functional data description. Application to curve discrimination. Neurocomputing, 73:1210–1221, March 2010.
  • Choi, T., Convergence of posterior distribution in the mixture of regressions. Journal of Nonparametric Statistics, 20(4):337–351, May 2008.
  • Cohen, S. and Le Pennec, E., Conditional density estimation by penalized likelihood model selection and applications. Technical report, INRIA, 2011.
  • Cohen, S. X. and Le Pennec, E., Partition-based conditional density estimation. ESAIM Probab. Stat., 17:672–697, 2013. ISSN 1292-8100. 10.1051/ps/2012017.
  • Dempster, A. P., Laird, N. M., and Rubin, D. B., Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society. Series B., 39(1), 1977.
  • Gassiat, E. and van Handel, R., The local geometry of finite mixtures. Trans. Amer. Math. Soc., 366(2):1047–1072, 2014.
  • Ge, Y. and Jiang, W., On consistency of bayesian inference with mixtures of logistic regression. Neural Computation, 18(1):224–243, January 2006.
  • Genovese, C. and Wasserman, L., Rates of convergence for the gaussian mixture sieve. The Annals of Statistics, 28(4):1105–1127, August 2000.
  • Huang, M., Li, R., and Wang, S., Nonparametric mixtures of regressions models. Journal of the American Statistical Association, 108(503):929–941, 2013.
  • Huang, M. and Yao, W., Mixture of regression models with varying mixing proportions: A semiparametric approach. J. Amer. Statist. Assoc., 107(498):711–724, 2012. ISSN 0162-1459. 10.1080/01621459.2012.682541.
  • Hunter, D. R. and Young, D. S., Semiparametric mixtures of regressions. J. Nonparametr. Stat., 24(1):19–38, 2012. ISSN 1048-5252. 10.1080/10485252.2011.608430.
  • Jordan, M. I. and Jacobs, R. A., Hierarchical mixtures of experts and the em algorithm. In Maria Marinaro and Pietro G. Morasso, editors, ICANN 94, pages 479–486. Springer London, 1994. ISBN 978-3-540-19887-1.
  • Kolaczyk, E. D., Ju, J., and Gopal, S., Multiscale, multigranular statistical image segmentation. Journal of the American Statistical Association, 100:1358–1369, 2005.
  • Lee, H. K. H., Consistency of posterior distributions for neural networks. Neural Networks, 13:629–642, July 2000.
  • Martin-Magniette, M. L., Mary-Huard, T., Bérard, C., and Robin, S., Chipmix: Mixture model of regressions for two-color chip-chip analysis. Bioinformatics, 24(16):i181–i186, 2008. 10.1093/bioinformatics/btn280.
  • Maugis, C. and Michel, B., A non asymptotic penalized criterion for gaussian mixture model selection. ESAIM Probability and Statistics, 2011.
  • Maugis, C. and Michel, B., Adaptive density estimation using finite gaussian mixtures. ESAIM Probability and Statistics, 2012. Accepted for publication.
  • McLachlan, G. and Peel, D., Finite Mixture Models. Wiley, 2000.
  • Rigollet, Ph., Kullback-Leibler aggregation and misspecified generalized linear models. The Annals of Statistics, 40(2):639–665, 2012. 10.1214/11-AOS961.
  • Van der Vaart, A. W. and Wellner, J. A., Weak Convergence and Empirical Processes. Springer, 1996.
  • Viele, K. and Tong, B., Modeling with mixtures of linear regressions. Stat. Comput., 12(4):315–330, 2002. ISSN 0960-3174. 10.1023/A:1020779827503.
  • Young, D. S., Mixtures of regressions with changepoints. Statistics and Computing, 24(2):265–281, 2014. ISSN 0960-3174. 10.1007/s11222-012-9369-x.
  • Young, D. S. and Hunter, D. R., Mixtures of regressions with predictor-dependent mixing proportions. Computational Statistics & Data Analysis, 54(10):2253–2266, 2010. ISSN 0167-9473.