Bernoulli

  • Bernoulli
  • Volume 12, Number 2 (2006), 351-370.

Adaptive density estimation using the blockwise Stein method

Philippe Rigollet

Full-text: Open access

Abstract

We study the problem of nonparametric estimation of a probability density of unknown smoothness in L2(R). Expressing mean integrated squared error (MISE) in the Fourier domain, we show that it is close to mean squared error in the Gaussian sequence model. Then applying a modified version of Stein's blockwise method, we obtain a linear monotone oracle inequality. Two consequences of this oracle inequality are that the proposed estimator is sharp minimax adaptive over a scale of Sobolev classes of densities, and that its MISE is asymptotically smaller than or equal to that of kernel density estimators with any bandwidth provided that the kernel belongs to a large class of functions including many standard kernels.

Article information

Source
Bernoulli, Volume 12, Number 2 (2006), 351-370.

Dates
First available in Project Euclid: 25 April 2006

Permanent link to this document
https://projecteuclid.org/euclid.bj/1145993978

Digital Object Identifier
doi:10.3150/bj/1145993978

Mathematical Reviews number (MathSciNet)
MR2218559

Zentralblatt MATH identifier
1098.62040

Keywords
adaptive density estimation blockwise Stein rule kernel oracle monotone oracle oracle inequalities

Citation

Rigollet, Philippe. Adaptive density estimation using the blockwise Stein method. Bernoulli 12 (2006), no. 2, 351--370. doi:10.3150/bj/1145993978. https://projecteuclid.org/euclid.bj/1145993978


Export citation

References

  • [1] Birgé, L. and Massart, P. (2001) Gaussian model selection. J. Eur. Math. Soc., 3, 203-268.
  • [2] Boiko, L.L. and Golubev, G.K. (2000) How to improve the nonparametric density estimation in SPLUS. Problems Inform. Transmission, 36, 354-361.
  • [3] Cai, T. (1999) Adaptive wavelet estimation: a block thresholding and oracle inequality approach. Ann. Statist., 27, 898-924.
  • [4] Cavalier, L. and Tsybakov, A.B. (2001) Penalized blockwise Stein´s method, monotone oracles and sharp adaptive estimation. Math. Methods Statist., 10, 247-282.
  • [5] Cavalier, L., Golubev, Yu., Picard, D. and Tsybakov, A.B. (2002) Oracle inequalities for inverse problems. Ann. Statist., 30, 843-874.
  • [6] Cline, D.B.H. (1988) Admissible kernel estimators of a multivariate density. Ann. Statist., 16, 1421- 1427.
  • [7] Dalelane, C. (2005) Exact minimax risk for density estimators in non-integer Sobolev classes. Preprint no. 979, Laboratoire de Probabilités et Modèles Aléatoires, Universités Paris 6 and Paris 7 (http://www.proba.jussieu.fr/mathdoc/preprints).
  • [8] Devroye, L. and Penrod, C.S. (1984) Distribution-free lower bounds in density estimation. Ann. Statist., 12, 1250-1262.
  • [9] Donoho, D.L. and Johnstone, I.M. (1994) Ideal spatial adaptation via wavelet shrinkage. Biometrika, 81, 425-455.
  • [10] Donoho, D.L. and Johnstone, I.M. (1995) Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc., 90, 1200-1224.
  • [11] Efroimovich, S.Yu. (1985) Nonparametric estimation of a density with unknown smoothness. Theory Probab. Appl., 30, 557-568.
  • [12] Efroimovich, S.Yu. and Pinsker, M.S. (1982) Estimation of square-integrable probability density of a random variable Problems Inform. Transmission, 18, 175-189.
  • [13] Efroimovich, S.Yu. and Pinsker, M.S. (1984) Learning algorithm for nonparametric filtering. Automat. Remote Control, 11, 1434-1440.
  • [14] Efromovich, S.Yu. (1999) Nonparametric Curve Estimation. New York: Springer-Verlag.
  • [15] Efromovich, S. (2000) On sharp adaptive estimation of multivariate curves. Math. Methods Statist., 9, 117-139.
  • [16] Efromovich, S. (2004a) Oracle inequalities for Efromovich-Pinsker blockwise estimates. Methodol. Comput. Appl. Probab., 6, 303-322.
  • [17] Efromovich, S. (2004b) Analysis of blockwise shrinkage wavelet estimates via lower bounds for nosignal setting. Ann. Inst. Statist. Math., 56, 205-223.
  • [18] Efromovich, S. (2005) Estimation of the density of regression errors. Ann. Statist., 33, 2194-2227.
  • [19] Ekeland, I. and Temam, R. (1974) Analyse convexe et problèmes variationnels. Paris: Dunod Gauthier- Villars.
  • [20] Goldenshluger, A., and Tsybakov, A. (2001) Adaptive prediction and estimation in linear regression with infinitely many parameters. Ann. Statist., 29, 1601-1619.
  • [21] Golubev, G.K. (1990) Quasi-linear estimates of signals in L2. Problems Inform. Transmission, 26, 15-20.
  • [22] Golubev, G.K. (1991) LAN in nonparametric estimation of functions and lower bounds for quadratic risks. Theory Probab. Appl., 36, 152-157.
  • [23] Golubev, G.K. (1992) Nonparametric estimation of smooth probability densties in L2. Problems Inform. Transmission, 28, 44-54.
  • [24] Golubev, G.K. and Levit B.Y. (1996) Distribution function estimation: adaptive smoothing. Math. Methods Statist., 5, 383-403.
  • [25] Golubev, G.K. and Nussbaum, M. (1992) Adaptive spline estimates in a nonparametric regression model. Theory Probab. Appl., 37, 521-529.
  • [26] Hall, P., Kerkyacharian, G. and Picard, D. (1998) Block thresholding rules for curve estimation using kernel and wavelet methods. Ann. Statist., 26, 922-942.
  • [27] Härdle, W., Kerkyacharian, G., Picard, D., and Tsybakov, A. (1998) Wavelets, Approximation, and Statistical Applications, Lecture Notes in Statist. 129. New York: Springer-Verlag.
  • [28] Kneip, A. (1994) Ordered linear smoothers. Ann. Statist., 22, 835-866.
  • [29] Li, K.-C. (1987) Asymptotic optimality of CP, CL, cross-validation and generalized cross-validation: Discrete index set. Ann. Statist., 15, 958-976.
  • [30] Lukacs, E. (1970). Characteristic Functions, 2nd edn. London: Griffin.
  • [31] Lukacs, E. (1983) Developments in Characteristic Function Theory. London: Griffin.
  • [32] Nemirovski, A. (2000) Topics in non-parametric statistics. In P. Bernard (ed.), Lectures on Probability Theory and Statistics. École d´Été de Probabilités de Saint-Flour XXVIII - 1998, Lecture Notes in Math. 1738, pp. 85-277. Berlin: Springer-Verlag.
  • [33] Polyak, B.T. and Tsybakov, A.B. (1992) A family of asymptotic optimal methods for choosing the estimate order in orthogonal series regression. Theory Probab. Appl., 37, 471-481.
  • [34] Rigollet, R. (2004) Adaptive density estimation using Stein´s blockwise method. Preprint no. 913, Laboratoire de Probabilités et Modèles Aléatoires, Universités Paris 6 and Paris 7 (http://www.proba.jussieu.fr/mathdoc/preprints).
  • [35] Schipper, M. (1996) Optimal rates and constants in L2-minimax estimation of probability density functions. Math. Methods Statist., 5, 253-274.
  • [36] Shibata, R. (1981) An optimal selection of regression variables. Biometrika, 68, 45-54.
  • [37] Stone, C.J. (1984) An asymptotically optimal window selection rule for kernel density estimates. Ann. Statist., 12, 1285-1297.
  • [38] Tsybakov, A. (2002) Discussion of 'Random rates in anisotropic regression´ by M Hoffmann and O Lepski. Ann. Statist., 30, 379-385.
  • [39] Tsybakov, A. (2004) Introduction à l´estimation non-paramétrique. Berlin: Springer-Verlag.
  • [40] Wegkamp, M.H. (1999) Quasi-universal bandwidth selection for kernel density estimators. Canad. J. Statist., 27, 409-420.