The Annals of Statistics

Adaptive estimation of and oracle inequalities for probability densities and characteristic functions

Sam Efromovich

Full-text: Open access


The theory of adaptive estimation and oracle inequalities for the case of Gaussian-shift–finite-interval experiments has made significant progress in recent years. In particular, sharp-minimax adaptive estimators and exact exponential-type oracle inequalities have been suggested for a vast set of functions including analytic and Sobolev with any positive index as well as for Efromovich–Pinsker and Stein blockwise-shrinkage estimators. Is it possible to obtain similar results for a more interesting applied problem of density estimation and/or the dual problem of characteristic function estimation? The answer is “yes.” In particular, the obtained results include exact exponential-type oracle inequalities which allow to consider, for the first time in the literature, a simultaneous sharp-minimax estimation of Sobolev densities with any positive index (not necessarily larger than 1/2), infinitely differentiable densities (including analytic, entire and stable), as well as of not absolutely integrable characteristic functions. The same adaptive estimator is also rate minimax over a familiar class of distributions with bounded spectrum where the density and the characteristic function can be estimated with the parametric rate.

Article information

Ann. Statist., Volume 36, Number 3 (2008), 1127-1155.

First available in Project Euclid: 26 May 2008

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation
Secondary: 62G20: Asymptotic properties

Blockwise shrinkage equivalence infinitely differentiable infinite support mean integrated squared error minimax nonparametric not absolutely integrable


Efromovich, Sam. Adaptive estimation of and oracle inequalities for probability densities and characteristic functions. Ann. Statist. 36 (2008), no. 3, 1127--1155. doi:10.1214/009053607000000965.

Export citation


  • [1] Abramowitz, M. and Stegun, I. (1964). Handbook of Mathematical Functions. Dover, New York.
  • [2] Balakrishnan, N. and Nevzorov, V. B. (2003). A Primer on Statistical Distributions. Wiley, Hoboken, NJ.
  • [3] Birgé, L. and Massart, P. (1997). From model selection to adaptive estimation. In Festschrift for Lucien Le Cam (D. Pollard, E. Torgersen and G. L. Yang, eds.) 55–87. Springer, New York.
  • [4] Brown, L. and Zhang, C.-H. (1998). Asymptotic nonequivalence of nonparametric experiments when the smoothness index is 1/2. Ann. Statist. 26 279–287.
  • [5] Brown, L., Carter, A., Low, M. and Zhang, C.-H. (2004). Equivalence theory for density estimation, Poisson processes and Gaussian white noise with drift. Ann. Statist. 32 2074–2097.
  • [6] Cai, T. (1999). Adaptive wavelet estimation: A block thresholding and oracle inequality approach. Ann. Statist. 27 898–924.
  • [7] Cai, T. (2002). On blockwise thresholding in wavelet regression: Adaptivity, block size and threshold level. Statist. Sinica 12 1241–1273.
  • [8] Cavalier, L. and Tsybakov, A. (2001). Penalized blockwise Stein’s method, monotone oracles and sharp adaptive estimation. Math. Methods Statist. 10 247–282.
  • [9] Chentsov, N. N. (1980). Statistical Decision Rules and Optimal Inference. Springer, New York.
  • [10] Chicken, E. and Cai, T. (2005). Block thresholding for density estimation: Local and global adaptivity. J. Multivariate Analysis 95 184–213.
  • [11] Dalelane, C. (2005). Exact minimax risk for density estimators in non-integer Sobolev classes. Preprint No. 979, Laboratoire de Probabilités et Modéles Aléatoires, Univ. Paris 6 and Paris 7. Available at
  • [12] DeCanditiis, D. and Vidakovic, B. (2004). Wavelet Bayesian block shrinkage via mixture of normal–inverse gamma priors. J. Comput. Graph. Statist. 13 383–398.
  • [13] de la Peña, V. and Montgomery-Smith, S. (1995). Decoupling inequalities for the tail probabilities of multivariate U-statistics. Ann. Probab. 23 806–816.
  • [14] Devroye, L. (1987). A Course in Density Estimation. Birkhäuser, Boston.
  • [15] Donoho, D. and Johnstone, D. (1995). Adaptation to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200–1224.
  • [16] Efromovich, S. and Pinsker, M. (1982). Estimation of a square integrable probability density of a random variable. Problems Inform. Transmission 18 19–38.
  • [17] Efromovich, S. and Pinsker, M. (1984). An adaptive algorithm of nonparametric filtering. Automat. Remote Control 11 58–65.
  • [18] Efromovich, S. (1985). Nonparametric estimation of a density with unknown smoothness. Theory Probab. Appl. 30 557–568.
  • [19] Efromovich, S. (1999). Nonparametric Curve Estimation: Methods, Theory and Applications. Springer, New York.
  • [20] Efromovich, S. (2001). Density estimation under random censorship and order restrictions: From asymptotic to small samples. J. Amer. Statist. Assoc. 96 667–685.
  • [21] Efromovich, S. (2004). Oracle inequalities for Efromovich–Pinsker blockwise estimates. Methodol. Comput. Appl. Probab. 6 303–322.
  • [22] Efromovich, S. (2004). Adaptive estimation of and oracle inequalities for probability densities and characteristic functions. Technical report, UNM.
  • [23] Efromovich, S. (2005). Estimation of the density of regression errors. Ann. Statist. 33 2194–2227.
  • [24] Efromovich, S. (2007). A lower bound oracle inequality for a blockwise–shrinkage estimate. J. Statist. Plann. Inference 137 176–183.
  • [25] Fan, J. and Gijbels, I. (1996). Local Polynomial Modeling and Its Applications. Chapman and Hall, London.
  • [26] Fan, J. and Yao, Q. (2003). Nonlinear Time Series: Nonparametric and Parametric Models. Springer, New York.
  • [27] Giné, E., Latala, R. and Zinn, J. (2000). Exponential and moment inequalities for U-statistics. In High Dimensional Probability II. Progress in Probab. 47 13–38. Birkhäuser, Boston.
  • [28] Giné, E., Koltchinskii, V. and Zinn, J. (2004). Weighted uniform consistency of kernel density estimators. Ann. Probab. 32 2570–2605.
  • [29] Golubev, G. K. (1992). Nonparametric estimation of smooth probability densities in L2. Problems Inform. Transmission 28 44–54.
  • [30] Golubev, G. K. and Levit, B. Y. (1996a). On the second order minimax estimation of distribution functions. Math. Methods Statist. 5 1–31.
  • [31] Golubev, G. K. and Levit, B. Y. (1996b). Asymptotically efficient estimation for analytic distributions. Math. Methods Statist. 5 357–368.
  • [32] Hendricks, H. (1990). Nonparametric estimation of a probability density on a Riemannian manifold using Fourier expansions. Ann. Statist. 18 832–849.
  • [33] Ibragimov, I. A. and Khasminskii, R. Z. (1982). Estimation of distribution density belonging to a class of entire functions. Theory Probab. Appl. 27 551–562.
  • [34] Ibragimov, I. A. and Khasminskii, R. Z. (1990). On density estimation in the view of Kolmogorov’s ideas in approximation theory. Ann. Statist. 18 999–1010.
  • [35] Ibragimov, I. A. (2001). Estimation of analytic functions. In State of the Art in Probability and Statistics (M. de Gunst, C. Klaasen and A. van der Vaart, eds.) 359–383. IMS, Beachwood, OH.
  • [36] Johnstone, I. M. (1998). Function Estimation in Gaussian Noise. Sequence Models. Draft of Monograph, Stanford Univ., California.
  • [37] Johnstone, I. M. and Silverman, B. (2005). Empirical Bayes selection of wavelet thresholds. Ann Statist. 33 1700–1752.
  • [38] Kawata, T. (1972). Fourier Analysis in Probability Theory. Academic Press, New York.
  • [39] Lepski, O. V. and Levit, B. Ya. (1998). Adaptive minimax estimation of infinitely differentiable functions. Math. Methods Statist. 7 123–156.
  • [40] Lukacs, E. (1960). Characteristic Functions. Hafner, London.
  • [41] Nemirovskii, A. S. (1999). Topics in Non-Parametric Statistics. Springer, Berlin.
  • [42] Nussbaum, M. (1996). Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Statist. 24 2399–2430.
  • [43] Oberhettinger, F. (1973). Fourier Transform of Distributions and Their Inverses. Academic Press, New York.
  • [44] Pinsker, M. S. (1980). Optimal filtering of square-integrable signals in Gaussian white noise. Problems Inform. Transmission 16 120–133.
  • [45] Prakasa Rao, B. L. S. (1983). Nonparametric Functional Estimation. Academic Press, New York.
  • [46] Rigollet, P. (2006). Adaptive density estimation using the blockwise Stein method. Bernoulli 12 351–370.
  • [47] Rudzkis, R. and Radavicius, M. (2005). Adaptive estimation of distribution density in the basis of algebraic polynomials. Theory Probab. Appl. 49 93–109.
  • [48] Samarov, A. (1992). Lower bound for the integral error of density function estimates. In Topics in Nonparametric Estimation (R. Z. Khasminskii, ed.) 1–6. Amer. Math. Soc., Providence, RI.
  • [49] Scott, D. (1992). Multivariate Density Estimation: Theory, Practice, and Visualization. Wiley, New York.
  • [50] Schipper, M. (1996). Optimal rates and constants in L2-minimax estimation of probability density functions. Math. Methods Statist. 5 253–274.
  • [51] Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis. Chapman and Hall, London.
  • [52] Tsybakov, A. (2002). Discussion of random rates in anisotropic regression. Ann. Statist. 30 379–385.
  • [53] Wasserman, L. (2005). All of Nonparametric Statistics. Springer, New York.
  • [54] Zhang, C.-H. (2005). General empirical Bayes wavelet methods and exactly adaptive minimax estimation. Ann. Statist. 33 54–100.
  • [55] Zolotarev, V. M. (1986). One-Dimensional Stable Distributions. Amer. Math. Soc., Providence, RI.