Adaptive estimation of and oracle inequalities for probability densities and characteristic functions

Adaptive estimation of and oracle inequalities for probability densities and characteristic functions

Sam Efromovich

Source: Ann. Statist. Volume 36, Number 3 (2008), 1127-1155.

Abstract

The theory of adaptive estimation and oracle inequalities for the case of Gaussian-shift–finite-interval experiments has made significant progress in recent years. In particular, sharp-minimax adaptive estimators and exact exponential-type oracle inequalities have been suggested for a vast set of functions including analytic and Sobolev with any positive index as well as for Efromovich–Pinsker and Stein blockwise-shrinkage estimators. Is it possible to obtain similar results for a more interesting applied problem of density estimation and/or the dual problem of characteristic function estimation? The answer is “yes.” In particular, the obtained results include exact exponential-type oracle inequalities which allow to consider, for the first time in the literature, a simultaneous sharp-minimax estimation of Sobolev densities with any positive index (not necessarily larger than 1/2), infinitely differentiable densities (including analytic, entire and stable), as well as of not absolutely integrable characteristic functions. The same adaptive estimator is also rate minimax over a familiar class of distributions with bounded spectrum where the density and the characteristic function can be estimated with the parametric rate.

First Page: Show

Primary Subjects: 62G05

Secondary Subjects: 62G20

Keywords: Blockwise shrinkage; equivalence; infinitely differentiable; infinite support; mean integrated squared error; minimax; nonparametric; not absolutely integrable

Full-text: Open access

Screen Optimized PDF File (265 KB)

Links and Identifiers

Permanent link to this document: http://projecteuclid.org/euclid.aos/1211819559
Digital Object Identifier: doi:10.1214/009053607000000965
Mathematical Reviews number (MathSciNet): MR2418652
Zentralblatt MATH identifier: 05294968

back to Table of Contents

References

[1] Abramowitz, M. and Stegun, I. (1964). Handbook of Mathematical Functions. Dover, New York.

[2] Balakrishnan, N. and Nevzorov, V. B. (2003). A Primer on Statistical Distributions. Wiley, Hoboken, NJ.

Mathematical Reviews (MathSciNet): MR1988562

Zentralblatt MATH: 1088.62020

[3] Birgé, L. and Massart, P. (1997). From model selection to adaptive estimation. In Festschrift for Lucien Le Cam (D. Pollard, E. Torgersen and G. L. Yang, eds.) 55–87. Springer, New York.

Mathematical Reviews (MathSciNet): MR1462939

[4] Brown, L. and Zhang, C.-H. (1998). Asymptotic nonequivalence of nonparametric experiments when the smoothness index is 1/2. Ann. Statist. 26 279–287.

Mathematical Reviews (MathSciNet): MR1611772

Digital Object Identifier: doi:10.1214/aos/1030563986

Project Euclid: euclid.aos/1030563986

Zentralblatt MATH: 0932.62061

[5] Brown, L., Carter, A., Low, M. and Zhang, C.-H. (2004). Equivalence theory for density estimation, Poisson processes and Gaussian white noise with drift. Ann. Statist. 32 2074–2097.

Mathematical Reviews (MathSciNet): MR2102503

Digital Object Identifier: doi:10.1214/009053604000000454

Project Euclid: euclid.aos/1098883782

Zentralblatt MATH: 1062.62083

[6] Cai, T. (1999). Adaptive wavelet estimation: A block thresholding and oracle inequality approach. Ann. Statist. 27 898–924.

Mathematical Reviews (MathSciNet): MR1724035

Digital Object Identifier: doi:10.1214/aos/1018031262

Project Euclid: euclid.aos/1018031262

Zentralblatt MATH: 0954.62047

[7] Cai, T. (2002). On blockwise thresholding in wavelet regression: Adaptivity, block size and threshold level. Statist. Sinica 12 1241–1273.

Mathematical Reviews (MathSciNet): MR1947074

Zentralblatt MATH: 1004.62036

[8] Cavalier, L. and Tsybakov, A. (2001). Penalized blockwise Stein’s method, monotone oracles and sharp adaptive estimation. Math. Methods Statist. 10 247–282.

Mathematical Reviews (MathSciNet): MR1867161

[9] Chentsov, N. N. (1980). Statistical Decision Rules and Optimal Inference. Springer, New York.

[10] Chicken, E. and Cai, T. (2005). Block thresholding for density estimation: Local and global adaptivity. J. Multivariate Analysis 95 184–213.

Mathematical Reviews (MathSciNet): MR2164124

Digital Object Identifier: doi:10.1016/j.jmva.2004.07.003

Zentralblatt MATH: 1064.62036

[11] Dalelane, C. (2005). Exact minimax risk for density estimators in non-integer Sobolev classes. Preprint No. 979, Laboratoire de Probabilités et Modéles Aléatoires, Univ. Paris 6 and Paris 7. Available at http://www.proba.jussieu.fr/mathdoc/preprints.

[12] DeCanditiis, D. and Vidakovic, B. (2004). Wavelet Bayesian block shrinkage via mixture of normal–inverse gamma priors. J. Comput. Graph. Statist. 13 383–398.

[13] de la Peña, V. and Montgomery-Smith, S. (1995). Decoupling inequalities for the tail probabilities of multivariate U-statistics. Ann. Probab. 23 806–816.

[14] Devroye, L. (1987). A Course in Density Estimation. Birkhäuser, Boston.

Mathematical Reviews (MathSciNet): MR891874

Zentralblatt MATH: 0617.62043

[15] Donoho, D. and Johnstone, D. (1995). Adaptation to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200–1224.

[16] Efromovich, S. and Pinsker, M. (1982). Estimation of a square integrable probability density of a random variable. Problems Inform. Transmission 18 19–38.

[17] Efromovich, S. and Pinsker, M. (1984). An adaptive algorithm of nonparametric filtering. Automat. Remote Control 11 58–65.

[18] Efromovich, S. (1985). Nonparametric estimation of a density with unknown smoothness. Theory Probab. Appl. 30 557–568.

Mathematical Reviews (MathSciNet): MR805304

[19] Efromovich, S. (1999). Nonparametric Curve Estimation: Methods, Theory and Applications. Springer, New York.

Mathematical Reviews (MathSciNet): MR1705298

Zentralblatt MATH: 0935.62039

[20] Efromovich, S. (2001). Density estimation under random censorship and order restrictions: From asymptotic to small samples. J. Amer. Statist. Assoc. 96 667–685.

Mathematical Reviews (MathSciNet): MR1946433

Digital Object Identifier: doi:10.1198/016214501753168334

JSTOR: links.jstor.org

Zentralblatt MATH: 1017.62029

[21] Efromovich, S. (2004). Oracle inequalities for Efromovich–Pinsker blockwise estimates. Methodol. Comput. Appl. Probab. 6 303–322.

Mathematical Reviews (MathSciNet): MR2073544

Digital Object Identifier: doi:10.1023/B:MCAP.0000026562.80429.48

Zentralblatt MATH: 1045.62020

[22] Efromovich, S. (2004). Adaptive estimation of and oracle inequalities for probability densities and characteristic functions. Technical report, UNM.

[23] Efromovich, S. (2005). Estimation of the density of regression errors. Ann. Statist. 33 2194–2227.

Mathematical Reviews (MathSciNet): MR2211084

Digital Object Identifier: doi:10.1214/009053605000000435

Project Euclid: euclid.aos/1132936561

Zentralblatt MATH: 1086.62053

[24] Efromovich, S. (2007). A lower bound oracle inequality for a blockwise–shrinkage estimate. J. Statist. Plann. Inference 137 176–183.

Mathematical Reviews (MathSciNet): MR2292849

Digital Object Identifier: doi:10.1016/j.jspi.2005.10.006

Zentralblatt MATH: 1098.62037

[25] Fan, J. and Gijbels, I. (1996). Local Polynomial Modeling and Its Applications. Chapman and Hall, London.

Mathematical Reviews (MathSciNet): MR1383587

Zentralblatt MATH: 0873.62037

[26] Fan, J. and Yao, Q. (2003). Nonlinear Time Series: Nonparametric and Parametric Models. Springer, New York.

Mathematical Reviews (MathSciNet): MR1964455

Zentralblatt MATH: 1014.62103

[27] Giné, E., Latala, R. and Zinn, J. (2000). Exponential and moment inequalities for U-statistics. In High Dimensional Probability II. Progress in Probab. 47 13–38. Birkhäuser, Boston.

Mathematical Reviews (MathSciNet): MR1857312

[28] Giné, E., Koltchinskii, V. and Zinn, J. (2004). Weighted uniform consistency of kernel density estimators. Ann. Probab. 32 2570–2605.

Mathematical Reviews (MathSciNet): MR2078551

Digital Object Identifier: doi:10.1214/009117904000000063

Project Euclid: euclid.aop/1091813624

Zentralblatt MATH: 1052.62034

[29] Golubev, G. K. (1992). Nonparametric estimation of smooth probability densities in L₂. Problems Inform. Transmission 28 44–54.

[30] Golubev, G. K. and Levit, B. Y. (1996a). On the second order minimax estimation of distribution functions. Math. Methods Statist. 5 1–31.

Mathematical Reviews (MathSciNet): MR1386823

[31] Golubev, G. K. and Levit, B. Y. (1996b). Asymptotically efficient estimation for analytic distributions. Math. Methods Statist. 5 357–368.

Mathematical Reviews (MathSciNet): MR1417678

[32] Hendricks, H. (1990). Nonparametric estimation of a probability density on a Riemannian manifold using Fourier expansions. Ann. Statist. 18 832–849.

Mathematical Reviews (MathSciNet): MR1056339

Digital Object Identifier: doi:10.1214/aos/1176347628

Project Euclid: euclid.aos/1176347628

Zentralblatt MATH: 0711.62036

[33] Ibragimov, I. A. and Khasminskii, R. Z. (1982). Estimation of distribution density belonging to a class of entire functions. Theory Probab. Appl. 27 551–562.

[34] Ibragimov, I. A. and Khasminskii, R. Z. (1990). On density estimation in the view of Kolmogorov’s ideas in approximation theory. Ann. Statist. 18 999–1010.

Mathematical Reviews (MathSciNet): MR1062695

Digital Object Identifier: doi:10.1214/aos/1176347736

Project Euclid: euclid.aos/1176347736

[35] Ibragimov, I. A. (2001). Estimation of analytic functions. In State of the Art in Probability and Statistics (M. de Gunst, C. Klaasen and A. van der Vaart, eds.) 359–383. IMS, Beachwood, OH.

Mathematical Reviews (MathSciNet): MR1836570

Digital Object Identifier: doi:10.1214/lnms/1215090078

[36] Johnstone, I. M. (1998). Function Estimation in Gaussian Noise. Sequence Models. Draft of Monograph, Stanford Univ., California.

[37] Johnstone, I. M. and Silverman, B. (2005). Empirical Bayes selection of wavelet thresholds. Ann Statist. 33 1700–1752.

Mathematical Reviews (MathSciNet): MR2166560

Digital Object Identifier: doi:10.1214/009053605000000345

Project Euclid: euclid.aos/1123250227

Zentralblatt MATH: 1078.62005

[38] Kawata, T. (1972). Fourier Analysis in Probability Theory. Academic Press, New York.

Mathematical Reviews (MathSciNet): MR464353

Zentralblatt MATH: 0271.60022

[39] Lepski, O. V. and Levit, B. Ya. (1998). Adaptive minimax estimation of infinitely differentiable functions. Math. Methods Statist. 7 123–156.

Mathematical Reviews (MathSciNet): MR1643256

[40] Lukacs, E. (1960). Characteristic Functions. Hafner, London.

Mathematical Reviews (MathSciNet): MR124075

Zentralblatt MATH: 0087.33605

[41] Nemirovskii, A. S. (1999). Topics in Non-Parametric Statistics. Springer, Berlin.

Mathematical Reviews (MathSciNet): MR1775640

Zentralblatt MATH: 0998.62033

[42] Nussbaum, M. (1996). Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Statist. 24 2399–2430.

Mathematical Reviews (MathSciNet): MR1425959

Digital Object Identifier: doi:10.1214/aos/1032181160

Project Euclid: euclid.aos/1032181160

[43] Oberhettinger, F. (1973). Fourier Transform of Distributions and Their Inverses. Academic Press, New York.

Mathematical Reviews (MathSciNet): MR352887

Zentralblatt MATH: 0306.65002

[44] Pinsker, M. S. (1980). Optimal filtering of square-integrable signals in Gaussian white noise. Problems Inform. Transmission 16 120–133.

Mathematical Reviews (MathSciNet): MR624591

Zentralblatt MATH: 0452.94003

[45] Prakasa Rao, B. L. S. (1983). Nonparametric Functional Estimation. Academic Press, New York.

Mathematical Reviews (MathSciNet): MR740865

Zentralblatt MATH: 0542.62025

[46] Rigollet, P. (2006). Adaptive density estimation using the blockwise Stein method. Bernoulli 12 351–370.

Mathematical Reviews (MathSciNet): MR2218559

Digital Object Identifier: doi:10.3150/bj/1145993978

Project Euclid: euclid.bj/1145993978

Zentralblatt MATH: 1098.62040

[47] Rudzkis, R. and Radavicius, M. (2005). Adaptive estimation of distribution density in the basis of algebraic polynomials. Theory Probab. Appl. 49 93–109.

Mathematical Reviews (MathSciNet): MR2141333

[48] Samarov, A. (1992). Lower bound for the integral error of density function estimates. In Topics in Nonparametric Estimation (R. Z. Khasminskii, ed.) 1–6. Amer. Math. Soc., Providence, RI.

Mathematical Reviews (MathSciNet): MR1191689

Zentralblatt MATH: 0773.62031

[49] Scott, D. (1992). Multivariate Density Estimation: Theory, Practice, and Visualization. Wiley, New York.

Mathematical Reviews (MathSciNet): MR1191168

Zentralblatt MATH: 0850.62006

[50] Schipper, M. (1996). Optimal rates and constants in L₂-minimax estimation of probability density functions. Math. Methods Statist. 5 253–274.

Mathematical Reviews (MathSciNet): MR1417672

[51] Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman and Hall, London.

Mathematical Reviews (MathSciNet): MR848134

Zentralblatt MATH: 0617.62042

[52] Tsybakov, A. (2002). Discussion of random rates in anisotropic regression. Ann. Statist. 30 379–385.

[53] Wasserman, L. (2005). All of Nonparametric Statistics. Springer, New York.

Mathematical Reviews (MathSciNet): MR2172729

Zentralblatt MATH: 02238053

[54] Zhang, C.-H. (2005). General empirical Bayes wavelet methods and exactly adaptive minimax estimation. Ann. Statist. 33 54–100.

Mathematical Reviews (MathSciNet): MR2157796

Digital Object Identifier: doi:10.1214/009053604000000995

Project Euclid: euclid.aos/1112967699

Zentralblatt MATH: 1064.62009

[55] Zolotarev, V. M. (1986). One-Dimensional Stable Distributions. Amer. Math. Soc., Providence, RI.

Mathematical Reviews (MathSciNet): MR854867

Zentralblatt MATH: 0589.60015

previous :: next