### Rates of convergence for the Gaussian mixture sieve

Christopher R. Genovese and Larry Wasserman
Source: Ann. Statist. Volume 28, Number 4 (2000), 1105-1127.

#### Abstract

Gaussian mixtures provide a convenient method of density estimation that lies somewhere between parametric models and kernel density estimators.When the number of components of the mixture is allowed to increase as sample size increases, the model is called a mixture sieve.We establish a bound on the rate of convergence in Hellinger distance for density estimation using the Gaussian mixture sieve assuming that the true density is itself a mixture of Gaussians; the underlying mixing measure of the true density is not necessarily assumed to have finite support. Computing the rate involves some delicate calculations since the size of the sieve—as measured by bracketing entropy—and the saturation rate, cannot be found using standard methods.When the mixing measure has compact support, using $k_n \sim n^{2/3}/(\log n)^{1/3}$ components in the mixture yields a rate of order $(\log n)^{(1+\eta)/6}/n^{1/6}$ for every $\eta > 0$. The rates depend heavilyon the tail behavior of the true density.The sensitivity to the tail behavior is dimin- ished byusing a robust sieve which includes a long-tailed component in the mixture.In the compact case,we obtain an improved rate of $(\log n/n)^{1/4}$. In the noncompact case, a spectrum of interesting rates arise depending on the thickness of the tails of the mixing measure.

First Page:
Primary Subjects: none
Full-text: Open access

Permanent link to this document: http://projecteuclid.org/euclid.aos/1015956709
Mathematical Reviews number (MathSciNet): MR1810921
Digital Object Identifier: doi:10.1214/aos/1015956709
Zentralblatt MATH identifier: 01828976

### References

Banfield, J. and Raftery, A. (1993). Model-based Gaussian and non-Gaussian clustering. Biometrics 49 803-821.
Mathematical Reviews (MathSciNet): MR1243494
Zentralblatt MATH: 0794.62034
Digital Object Identifier: doi:10.2307/2532201
Barron, A. and Yang, Y. (1995). An asymptotic property of model selection criteria. Technical report, Dept. Statistics, Yale Univ.
Chen, J. (1995). Optimal rate of convergence for finite mixtures models. Ann. Statist. 23 221-233.
Mathematical Reviews (MathSciNet): MR1331665
Zentralblatt MATH: 0821.62023
Digital Object Identifier: doi:10.1214/aos/1176324464
Project Euclid: euclid.aos/1176324464
Escobar, M. and West, M. (1995). Bayesian density estimation and inference using mixtures. J. Amer. Statist. Assoc. 90 577-588.
Mathematical Reviews (MathSciNet): MR96d:62054
Zentralblatt MATH: 0826.62021
Digital Object Identifier: doi:10.2307/2291069
Gemen, S. and Hwang, C. (1982). Nonparametric maximum likelihood estimation bythe method of sieves. Ann. Statist. 10 401-414.
Mathematical Reviews (MathSciNet): MR653512
Zentralblatt MATH: 0494.62041
Digital Object Identifier: doi:10.1214/aos/1176345782
Project Euclid: euclid.aos/1176345782
Ghosal, S. and van der Vaart, A. (2000). Rates of convergence for Bayes and maximum likelihood estimation for mixtures of normal densities. Unpublished manuscript.
Grenander, U. (1981). Abstract Inference. Wiley, New York.
Mathematical Reviews (MathSciNet): MR82g:62004
Hall, P. (1987). On Kullback-Leibler loss and densityestimation. Ann. Statist. 15 1491-1519.
Mathematical Reviews (MathSciNet): MR913570
Zentralblatt MATH: 0678.62045
Digital Object Identifier: doi:10.1214/aos/1176350606
Project Euclid: euclid.aos/1176350606
Li, J. (1999). Estimation of mixtures models. Ph.D. dissertation, Dept. Statistics. Yale Univ.
Li, J. and Barron, A. (1999). Mixture densityestimation. Preprint.
Lindsay, B. (1995). Mixture Models: Theory, Geometry and Applications. IMS, Hayward, CA.
McLachlan, G. and Basford, K. (1988). Mixture Models: Inference and Applications to Clustering. Dekker, New York.
Mathematical Reviews (MathSciNet): MR926484
Priebe, C. (1994). Adaptive mixtures. J. Amer. Statist. Assoc. 89 796-806.
Mathematical Reviews (MathSciNet): MR1294725
Zentralblatt MATH: 0825.62445
Digital Object Identifier: doi:10.2307/2290905
Robert, C. (1996). Mixtures of distributions: inference and estimation. In Markov Chain Monte Carlo in Practice (W. Gilks, S. Richardson, D. Spiegelhalter, eds.) 441-464. Chapman and Hall, London.
Mathematical Reviews (MathSciNet): MR1397974
Zentralblatt MATH: 0849.62013
Roeder, K. and Wasserman, L. (1997). Practical Bayesian density estimation using mixtures of normals. J. Amer. Statist. Assoc. 92 894-902.
Mathematical Reviews (MathSciNet): MR98k:62038
Zentralblatt MATH: 0889.62021
Digital Object Identifier: doi:10.2307/2965553
Roeder, K. (1992). Semiparametric estimation of normal mixture densities. Ann. Statist. 20 929-943.
Mathematical Reviews (MathSciNet): MR93e:62116
Zentralblatt MATH: 0746.62044
Digital Object Identifier: doi:10.1214/aos/1176348664
Project Euclid: euclid.aos/1176348664
Tong, B., and Viele, K. (1998). Mixtures of normal linear regressions. Technical report, Univ. Kentucky.
van de Geer, S. (1996). Rates of convergence for the maximum likelihood estimator in mixture models. Nonparametric Statist. 6 293-310.
Mathematical Reviews (MathSciNet): MR97e:62057
Zentralblatt MATH: 0872.62039
Digital Object Identifier: doi:10.1080/10485259608832677
van der Vaart, A. and Wellner, J. (1996). Weak Convergence and Empirical Processes. Springer, New York.
Mathematical Reviews (MathSciNet): MR97g:60035
Zentralblatt MATH: 0862.60002
Wong, W. and Shen, X. (1995). Probabilityinequalities for likelihood ratios and convergence rates of sieve MLEs. Ann. Statist. 23 339-362.
Mathematical Reviews (MathSciNet): MR1332570
Zentralblatt MATH: 0829.62002
Digital Object Identifier: doi:10.1214/aos/1176324524
Project Euclid: euclid.aos/1176324524