Bernoulli
- Bernoulli
- Volume 12, Number 2 (2006), 351-370.
Adaptive density estimation using the blockwise Stein method
Full-text: Open access
Abstract
We study the problem of nonparametric estimation of a probability density of unknown smoothness in L2(R). Expressing mean integrated squared error (MISE) in the Fourier domain, we show that it is close to mean squared error in the Gaussian sequence model. Then applying a modified version of Stein's blockwise method, we obtain a linear monotone oracle inequality. Two consequences of this oracle inequality are that the proposed estimator is sharp minimax adaptive over a scale of Sobolev classes of densities, and that its MISE is asymptotically smaller than or equal to that of kernel density estimators with any bandwidth provided that the kernel belongs to a large class of functions including many standard kernels.
Article information
Source
Bernoulli, Volume 12, Number 2 (2006), 351-370.
Dates
First available in Project Euclid: 25 April 2006
Permanent link to this document
https://projecteuclid.org/euclid.bj/1145993978
Digital Object Identifier
doi:10.3150/bj/1145993978
Mathematical Reviews number (MathSciNet)
MR2218559
Zentralblatt MATH identifier
1098.62040
Keywords
adaptive density estimation blockwise Stein rule kernel oracle monotone oracle oracle inequalities
Citation
Rigollet, Philippe. Adaptive density estimation using the blockwise Stein method. Bernoulli 12 (2006), no. 2, 351--370. doi:10.3150/bj/1145993978. https://projecteuclid.org/euclid.bj/1145993978
References
- [1] Birgé, L. and Massart, P. (2001) Gaussian model selection. J. Eur. Math. Soc., 3, 203-268.
- [2] Boiko, L.L. and Golubev, G.K. (2000) How to improve the nonparametric density estimation in SPLUS. Problems Inform. Transmission, 36, 354-361.Mathematical Reviews (MathSciNet): MR1813655
- [3] Cai, T. (1999) Adaptive wavelet estimation: a block thresholding and oracle inequality approach. Ann. Statist., 27, 898-924.Mathematical Reviews (MathSciNet): MR1724035
Zentralblatt MATH: 0954.62047
Digital Object Identifier: doi:10.1214/aos/1018031262
Project Euclid: euclid.aos/1018031262 - [4] Cavalier, L. and Tsybakov, A.B. (2001) Penalized blockwise Stein´s method, monotone oracles and sharp adaptive estimation. Math. Methods Statist., 10, 247-282.
- [5] Cavalier, L., Golubev, Yu., Picard, D. and Tsybakov, A.B. (2002) Oracle inequalities for inverse problems. Ann. Statist., 30, 843-874.Mathematical Reviews (MathSciNet): MR1922543
Zentralblatt MATH: 1029.62032
Digital Object Identifier: doi:10.1214/aos/1028674843
Project Euclid: euclid.aos/1028674843 - [6] Cline, D.B.H. (1988) Admissible kernel estimators of a multivariate density. Ann. Statist., 16, 1421- 1427.Mathematical Reviews (MathSciNet): MR964931
Zentralblatt MATH: 0653.62027
Digital Object Identifier: doi:10.1214/aos/1176351046
Project Euclid: euclid.aos/1176351046 - [7] Dalelane, C. (2005) Exact minimax risk for density estimators in non-integer Sobolev classes. Preprint no. 979, Laboratoire de Probabilités et Modèles Aléatoires, Universités Paris 6 and Paris 7 (http://www.proba.jussieu.fr/mathdoc/preprints).URL: Link to item
- [8] Devroye, L. and Penrod, C.S. (1984) Distribution-free lower bounds in density estimation. Ann. Statist., 12, 1250-1262.Mathematical Reviews (MathSciNet): MR760686
Zentralblatt MATH: 0551.62024
Digital Object Identifier: doi:10.1214/aos/1176346790
Project Euclid: euclid.aos/1176346790 - [9] Donoho, D.L. and Johnstone, I.M. (1994) Ideal spatial adaptation via wavelet shrinkage. Biometrika, 81, 425-455.Mathematical Reviews (MathSciNet): MR1311089
Zentralblatt MATH: 0815.62019
Digital Object Identifier: doi:10.1093/biomet/81.3.425
JSTOR: links.jstor.org - [10] Donoho, D.L. and Johnstone, I.M. (1995) Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc., 90, 1200-1224.Mathematical Reviews (MathSciNet): MR1379464
Zentralblatt MATH: 0869.62024
Digital Object Identifier: doi:10.2307/2291512
JSTOR: links.jstor.org - [11] Efroimovich, S.Yu. (1985) Nonparametric estimation of a density with unknown smoothness. Theory Probab. Appl., 30, 557-568.Mathematical Reviews (MathSciNet): MR805304
- [12] Efroimovich, S.Yu. and Pinsker, M.S. (1982) Estimation of square-integrable probability density of a random variable Problems Inform. Transmission, 18, 175-189.Mathematical Reviews (MathSciNet): MR711898
- [13] Efroimovich, S.Yu. and Pinsker, M.S. (1984) Learning algorithm for nonparametric filtering. Automat. Remote Control, 11, 1434-1440.
- [14] Efromovich, S.Yu. (1999) Nonparametric Curve Estimation. New York: Springer-Verlag.Mathematical Reviews (MathSciNet): MR1705298
- [15] Efromovich, S. (2000) On sharp adaptive estimation of multivariate curves. Math. Methods Statist., 9, 117-139.
- [16] Efromovich, S. (2004a) Oracle inequalities for Efromovich-Pinsker blockwise estimates. Methodol. Comput. Appl. Probab., 6, 303-322.Mathematical Reviews (MathSciNet): MR2073544
Zentralblatt MATH: 1045.62020
Digital Object Identifier: doi:10.1023/B:MCAP.0000026562.80429.48 - [17] Efromovich, S. (2004b) Analysis of blockwise shrinkage wavelet estimates via lower bounds for nosignal setting. Ann. Inst. Statist. Math., 56, 205-223.Mathematical Reviews (MathSciNet): MR2067153
Zentralblatt MATH: 1056.62048
Digital Object Identifier: doi:10.1007/BF02530542 - [18] Efromovich, S. (2005) Estimation of the density of regression errors. Ann. Statist., 33, 2194-2227.Mathematical Reviews (MathSciNet): MR2211084
Zentralblatt MATH: 1086.62053
Digital Object Identifier: doi:10.1214/009053605000000435
Project Euclid: euclid.aos/1132936561 - [19] Ekeland, I. and Temam, R. (1974) Analyse convexe et problèmes variationnels. Paris: Dunod Gauthier- Villars.
- [20] Goldenshluger, A., and Tsybakov, A. (2001) Adaptive prediction and estimation in linear regression with infinitely many parameters. Ann. Statist., 29, 1601-1619.Mathematical Reviews (MathSciNet): MR1891740
Zentralblatt MATH: 1043.62076
Digital Object Identifier: doi:10.1214/aos/1015345956
Project Euclid: euclid.aos/1015345956 - [21] Golubev, G.K. (1990) Quasi-linear estimates of signals in L2. Problems Inform. Transmission, 26, 15-20.
- [22] Golubev, G.K. (1991) LAN in nonparametric estimation of functions and lower bounds for quadratic risks. Theory Probab. Appl., 36, 152-157.Mathematical Reviews (MathSciNet): MR1109023
- [23] Golubev, G.K. (1992) Nonparametric estimation of smooth probability densties in L2. Problems Inform. Transmission, 28, 44-54.Mathematical Reviews (MathSciNet): MR1163140
- [24] Golubev, G.K. and Levit B.Y. (1996) Distribution function estimation: adaptive smoothing. Math. Methods Statist., 5, 383-403.
- [25] Golubev, G.K. and Nussbaum, M. (1992) Adaptive spline estimates in a nonparametric regression model. Theory Probab. Appl., 37, 521-529.Mathematical Reviews (MathSciNet): MR1214361
- [26] Hall, P., Kerkyacharian, G. and Picard, D. (1998) Block thresholding rules for curve estimation using kernel and wavelet methods. Ann. Statist., 26, 922-942.Mathematical Reviews (MathSciNet): MR1635418
Zentralblatt MATH: 0929.62040
Digital Object Identifier: doi:10.1214/aos/1024691082
Project Euclid: euclid.aos/1024691082 - [27] Härdle, W., Kerkyacharian, G., Picard, D., and Tsybakov, A. (1998) Wavelets, Approximation, and Statistical Applications, Lecture Notes in Statist. 129. New York: Springer-Verlag.
- [28] Kneip, A. (1994) Ordered linear smoothers. Ann. Statist., 22, 835-866.Mathematical Reviews (MathSciNet): MR1292543
Zentralblatt MATH: 0815.62022
Digital Object Identifier: doi:10.1214/aos/1176325498
Project Euclid: euclid.aos/1176325498 - [29] Li, K.-C. (1987) Asymptotic optimality of CP, CL, cross-validation and generalized cross-validation: Discrete index set. Ann. Statist., 15, 958-976.Mathematical Reviews (MathSciNet): MR902239
Zentralblatt MATH: 0653.62037
Digital Object Identifier: doi:10.1214/aos/1176350486
Project Euclid: euclid.aos/1176350486 - [30] Lukacs, E. (1970). Characteristic Functions, 2nd edn. London: Griffin.
- [31] Lukacs, E. (1983) Developments in Characteristic Function Theory. London: Griffin.
- [32] Nemirovski, A. (2000) Topics in non-parametric statistics. In P. Bernard (ed.), Lectures on Probability Theory and Statistics. École d´Été de Probabilités de Saint-Flour XXVIII - 1998, Lecture Notes in Math. 1738, pp. 85-277. Berlin: Springer-Verlag.
- [33] Polyak, B.T. and Tsybakov, A.B. (1992) A family of asymptotic optimal methods for choosing the estimate order in orthogonal series regression. Theory Probab. Appl., 37, 471-481.Mathematical Reviews (MathSciNet): MR1214355
- [34] Rigollet, R. (2004) Adaptive density estimation using Stein´s blockwise method. Preprint no. 913, Laboratoire de Probabilités et Modèles Aléatoires, Universités Paris 6 and Paris 7 (http://www.proba.jussieu.fr/mathdoc/preprints).URL: Link to item
- [35] Schipper, M. (1996) Optimal rates and constants in L2-minimax estimation of probability density functions. Math. Methods Statist., 5, 253-274.
- [36] Shibata, R. (1981) An optimal selection of regression variables. Biometrika, 68, 45-54.Mathematical Reviews (MathSciNet): MR614940
Zentralblatt MATH: 0464.62054
Digital Object Identifier: doi:10.1093/biomet/68.1.45
JSTOR: links.jstor.org - [37] Stone, C.J. (1984) An asymptotically optimal window selection rule for kernel density estimates. Ann. Statist., 12, 1285-1297.Mathematical Reviews (MathSciNet): MR760688
Zentralblatt MATH: 0599.62052
Digital Object Identifier: doi:10.1214/aos/1176346792
Project Euclid: euclid.aos/1176346792 - [38] Tsybakov, A. (2002) Discussion of 'Random rates in anisotropic regression´ by M Hoffmann and O Lepski. Ann. Statist., 30, 379-385.
- [39] Tsybakov, A. (2004) Introduction à l´estimation non-paramétrique. Berlin: Springer-Verlag.
- [40] Wegkamp, M.H. (1999) Quasi-universal bandwidth selection for kernel density estimators. Canad. J. Statist., 27, 409-420.Mathematical Reviews (MathSciNet): MR1704442
Digital Object Identifier: doi:10.2307/3315649
JSTOR: links.jstor.org
- You have access to this content.
- You have partial access to this content.
- You do not have access to this content.
More like this
- Adaptive Prediction and Estimation in Linear Regression with
Infinitely Many Parameters
Goldenshluger, A. and Tsybakov, A., The Annals of Statistics, 2001 - Adaptive estimation of and oracle inequalities for probability densities and characteristic functions
Efromovich, Sam, The Annals of Statistics, 2008 - Estimation of the density of regression errors
Efromovich, Sam, The Annals of Statistics, 2005
- Adaptive Prediction and Estimation in Linear Regression with
Infinitely Many Parameters
Goldenshluger, A. and Tsybakov, A., The Annals of Statistics, 2001 - Adaptive estimation of and oracle inequalities for probability densities and characteristic functions
Efromovich, Sam, The Annals of Statistics, 2008 - Estimation of the density of regression errors
Efromovich, Sam, The Annals of Statistics, 2005 - Formulae for Mean Integrated Squared Error of Nonlinear Wavelet-Based Density Estimators
Hall, Peter and Patil, Prakash, The Annals of Statistics, 1995 - A Risk Bound in Sobolev Class Regression
Golubev, Grigori K. and Nussbaum, Michael, The Annals of Statistics, 1990 - Adaptive demixing in Poisson mixture models
Hengartner, Nicolas W., The Annals of Statistics, 1997 - Adaptive estimation over anisotropic functional classes via oracle approach
Lepski, Oleg, The Annals of Statistics, 2015 - Adaptivity and optimality of the monotone least-squares estimator
Cator, Eric, Bernoulli, 2011 - Using Stopping Rules to Bound the Mean Integrated Squared Error in Density Estimation
Martinsek, Adam T., The Annals of Statistics, 1992 - Adaptive density estimation in the pile-up model involving measurement errors
Comte, Fabienne and Rebafka, Tabea, Electronic Journal of Statistics, 2012