Annals of Statistics

Anti-concentration and honest, adaptive confidence bands

Victor Chernozhukov, Denis Chetverikov, and Kengo Kato

Full-text: Open access


Modern construction of uniform confidence bands for nonparametric densities (and other functions) often relies on the classical Smirnov–Bickel–Rosenblatt (SBR) condition; see, for example, Giné and Nickl [Probab. Theory Related Fields 143 (2009) 569–596]. This condition requires the existence of a limit distribution of an extreme value type for the supremum of a studentized empirical process (equivalently, for the supremum of a Gaussian process with the same covariance function as that of the studentized empirical process). The principal contribution of this paper is to remove the need for this classical condition. We show that a considerably weaker sufficient condition is derived from an anti-concentration property of the supremum of the approximating Gaussian process, and we derive an inequality leading to such a property for separable Gaussian processes. We refer to the new condition as a generalized SBR condition. Our new result shows that the supremum does not concentrate too fast around any value.

We then apply this result to derive a Gaussian multiplier bootstrap procedure for constructing honest confidence bands for nonparametric density estimators (this result can be applied in other nonparametric problems as well). An essential advantage of our approach is that it applies generically even in those cases where the limit distribution of the supremum of the studentized empirical process does not exist (or is unknown). This is of particular importance in problems where resolution levels or other tuning parameters have been chosen in a data-driven fashion, which is needed for adaptive constructions of the confidence bands. Finally, of independent interest is our introduction of a new, practical version of Lepski’s method, which computes the optimal, nonconservative resolution levels via a Gaussian multiplier bootstrap method.

Article information

Ann. Statist., Volume 42, Number 5 (2014), 1787-1818.

First available in Project Euclid: 11 September 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation 62G15: Tolerance and confidence regions

Anti-concentration of separable Gaussian processes honest confidence bands Lepski’s method multiplier method non-Donsker empirical processes


Chernozhukov, Victor; Chetverikov, Denis; Kato, Kengo. Anti-concentration and honest, adaptive confidence bands. Ann. Statist. 42 (2014), no. 5, 1787--1818. doi:10.1214/14-AOS1235.

Export citation


  • [1] Bickel, P. J. and Rosenblatt, M. (1973). On some global measures of the deviations of density function estimates. Ann. Statist. 1 1071–1095.
  • [2] Birgé, L. (2001). An alternative point of view on Lepski’s method. In State of the Art in Probability and Statistics (Leiden, 1999). Institute of Mathematical Statistics Lecture Notes—Monograph Series 36 113–133. IMS, Beachwood, OH.
  • [3] Bissantz, N., Dümbgen, L., Holzmann, H. and Munk, A. (2007). Non-parametric confidence bands in deconvolution density estimation. J. R. Stat. Soc. Ser. B Stat. Methodol. 69 483–506.
  • [4] Bull, A. D. (2012). Honest adaptive confidence bands and self-similar functions. Electron. J. Stat. 6 1490–1516.
  • [5] Bull, A. D. (2013). A Smirnov–Bickel–Rosenblatt theorem for compactly-supported wavelets. Constr. Approx. 37 295–309.
  • [6] Cai, T. T. and Low, M. G. (2004). An adaptation theory for nonparametric confidence intervals. Ann. Statist. 32 1805–1840.
  • [7] Chernozhukov, V., Chetverikov, D. and Kato, K. (2012). Gaussian approximation of suprema of empirical processes. Preprint. Available at arXiv:1212.6885v2.
  • [8] Chernozhukov, V., Chetverikov, D. and Kato, K. (2014). Comparison and anti-concentration bounds for maxima of Gaussian random vectors. Probab. Theory Related Fields. To appear. Available at arXiv:1301.4807v3.
  • [9] Chernozhukov, V., Chetverikov, D. and Kato, K. (2014). Supplement to “Anti-concentration and honest, adaptive confidence bands.” DOI:10.1214/14-AOS1235SUPP.
  • [10] Chernozhukov, V., Lee, S. and Rosen, A. M. (2013). Intersection bounds: Estimation and inference. Econometrica 81 667–737.
  • [11] Chetverikov, D. (2012). Testing regression monotonicity in econometric models. Available at arXiv:1212.6757.
  • [12] Claeskens, G. and Van Keilegom, I. (2003). Bootstrap confidence bands for regression curves and their derivatives. Ann. Statist. 31 1852–1884.
  • [13] Daubechies, I. (1992). Ten Lectures on Wavelets. CBMS-NSF Regional Conference Series in Applied Mathematics 61. SIAM, Philadelphia, PA.
  • [14] Dudley, R. M. (1999). Uniform Central Limit Theorems. Cambridge Univ. Press, Cambridge.
  • [15] Giné, E. and Guillou, A. (2001). A law of the iterated logarithm for kernel density estimators in the presence of censoring. Ann. Inst. Henri Poincaré Probab. Stat. 37 503–522.
  • [16] Giné, E. and Guillou, A. (2002). Rates of strong uniform consistency for multivariate kernel density estimators. Ann. Inst. Henri Poincaré Probab. Stat. 38 907–921.
  • [17] Giné, E., Güntürk, C. S. and Madych, W. R. (2011). On the periodized square of $L^2$ cardinal splines. Exp. Math. 20 177–188.
  • [18] Giné, E., Koltchinskii, V. and Sakhanenko, L. (2004). Kernel density estimators: Convergence in distribution for weighted sup-norms. Probab. Theory Related Fields 130 167–198.
  • [19] Giné, E. and Nickl, R. (2009). An exponential inequality for the distribution function of the kernel density estimator, with applications to adaptive estimation. Probab. Theory Related Fields 143 569–596.
  • [20] Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122–1170.
  • [21] Giné, E. and Nickl, R. (2010). Adaptive estimation of a distribution function and its density in sup-norm loss by wavelet and spline projections. Bernoulli 16 1137–1163.
  • [22] Giné, E. and Zinn, J. (1984). Some limit theorems for empirical processes. Ann. Probab. 12 929–998.
  • [23] Giné, E. and Zinn, J. (1990). Bootstrapping general empirical measures. Ann. Probab. 18 851–869.
  • [24] Hall, P. (1991). On convergence rates of suprema. Probab. Theory Related Fields 89 447–455.
  • [25] Hall, P. and Horowitz, J. (2013). A simple bootstrap method for constructing nonparametric confidence bands for functions. Ann. Statist. 41 1892–1921.
  • [26] Härdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. (1998). Wavelets, Approximation, and Statistical Applications. Springer, New York.
  • [27] Hoffmann, M. and Nickl, R. (2011). On adaptive inference and confidence bands. Ann. Statist. 39 2383–2409.
  • [28] Komlós, J., Major, P. and Tusnády, G. (1975). An approximation of partial sums of independent $\mathrmRV$’s and the sample $\mathrmDF$. I. Z. Wahrsch. Verw. Gebiete 32 111–131.
  • [29] Ledoux, M. and Talagrand, M. (1988). Un critère sur les petites boules dans le théorème limite central. Probab. Theory Related Fields 77 29–47.
  • [30] Lepskiĭ, O. V. (1991). Asymptotically minimax adaptive estimation. I. Upper bounds. Optimally adaptive estimates. Theory Probab. Appl. 36 682–697.
  • [31] Li, K.-C. (1989). Honest confidence regions for nonparametric regression. Ann. Statist. 17 1001–1008.
  • [32] Lounici, K. and Nickl, R. (2011). Global uniform risk bounds for wavelet deconvolution estimators. Ann. Statist. 39 201–231.
  • [33] Low, M. G. (1997). On nonparametric confidence intervals. Ann. Statist. 25 2547–2554.
  • [34] Massart, P. (2000). About the constants in Talagrand’s concentration inequalities for empirical processes. Ann. Probab. 28 863–884.
  • [35] Massart, P. (2007). Concentration Inequalities and Model Selection. Springer, Berlin.
  • [36] Picard, D. and Tribouley, K. (2000). Adaptive confidence interval for pointwise curve estimation. Ann. Statist. 28 298–335.
  • [37] Præstgaard, J. and Wellner, J. A. (1993). Exchangeably weighted bootstraps of the general empirical process. Ann. Probab. 21 2053–2086.
  • [38] Rio, E. (1994). Local invariance principles and their application to density estimation. Probab. Theory Related Fields 98 21–45.
  • [39] Rudelson, M. and Vershynin, R. (2009). Smallest singular value of a random rectangular matrix. Comm. Pure Appl. Math. 62 1707–1739.
  • [40] Smirnov, N. V. (1950). On the construction of confidence regions for the density of distribution of random variables. Doklady Akad. Nauk SSSR (N.S.) 74 189–191.
  • [41] Talagrand, M. (1996). New concentration inequalities in product spaces. Invent. Math. 126 505–563.
  • [42] Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation. Springer, New York.
  • [43] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
  • [44] Wasserman, L. (2006). All of Nonparametric Statistics. Springer, New York.
  • [45] Xia, Y. (1998). Bias-corrected confidence bands in nonparametric regression. J. R. Stat. Soc. Ser. B Stat. Methodol. 60 797–811.

Supplemental materials

  • Supplementary material: Supplement to “Anti-concentration and honest, adaptive confidence bands”. This supplemental file contains additional proofs omitted in the main text, some results regarding nonwavelet projection kernel estimators, and a small-scale simulation study.