The Annals of Statistics

Nonparametric estimation by convex programming

Anatoli B. Juditsky and Arkadi S. Nemirovski

Full-text: Open access


The problem we concentrate on is as follows: given (1) a convex compact set X in ℝn, an affine mapping xA(x), a parametric family {pμ(⋅)} of probability densities and (2) N i.i.d. observations of the random variable ω, distributed with the density pA(x)(⋅) for some (unknown) xX, estimate the value gTx of a given linear form at x.

For several families {pμ(⋅)} with no additional assumptions on X and A, we develop computationally efficient estimation routines which are minimax optimal, within an absolute constant factor. We then apply these routines to recovering x itself in the Euclidean norm.

Article information

Ann. Statist., Volume 37, Number 5A (2009), 2278-2300.

First available in Project Euclid: 15 July 2009

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression
Secondary: 62G15: Tolerance and confidence regions 62G07: Density estimation

Estimation of linear functional minimax estimation oracle inequalities convex optimization PE tomography


Juditsky, Anatoli B.; Nemirovski, Arkadi S. Nonparametric estimation by convex programming. Ann. Statist. 37 (2009), no. 5A, 2278--2300. doi:10.1214/08-AOS654.

Export citation


  • [1] Ben-Tal, A. and Nemirovski, A. (2001). Lectures on Modern Convex Optimization: Analysis, Algorithms and Engineering Applications. SIAM, Philadelphia.
  • [2] Birgé, L. (1984). Sur un théorème de minimax et son application aux tests. (French) Probab. Math. Statist. 3 259–282.
  • [3] Birgé, L. (1983). Approximation dans les espaces métriques et théorie de l’estimation. Z. Wahrsch. Verw. Gebiete 65 181–237.
  • [4] Cai, T. and Low, M. (2003). A note on nonparametric estimation of linear functionals. Ann. Statist. 31 1140–1153.
  • [5] Donoho, D. (1995). Statistical estimation and optimal recovery. Ann. Statist. 22 238–270.
  • [6] Donoho, D. and Liu, R. (1987). Geometrizing Rates of Convergence. I. Technical Report 137a, Dept. Statistics, Univ. California, Berkeley.
  • [7] Donoho, D. and Liu, R. (1991). Geometrizing rates of convergence. II. Ann. Statist. 19 633–667.
  • [8] Donoho, D., Liu, R. and MacGibbon, B. (1990). Minimax risk over hyperrectangles, and implications. Ann. Statist. 18 1416–1437.
  • [9] Donoho, D. and Low, M. (1992). Renormalization exponents and optimal pointwise rates of convergence. Ann. Statist. 20 944–970.
  • [10] Eubank, R. (1988). Spline Smoothing and Nonparametric Regression. Dekker, New York.
  • [11] Goldenshluger, A. and Nemirovski, A. (1997). On spatially adaptive estimation of nonparametric regression. Math. Methods Statist. 6 135–170.
  • [12] Härdle, W. (1990). Applied Nonparametric Regression. ES Monograph Series 19. Cambridge Univ. Press, Cambridge, UK.
  • [13] Härdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. B. (1998). Wavelets, Approximation and Statistical Applications. Lecture Notes in Statistics 129. Springer, New York.
  • [14] Hiriart-Urruty, J. B. and Lemarechal, C. (1993). Convex Analysis and Minimization Algorithms I: Fundamentals. Springer, Berlin.
  • [15] Ibragimov, I. A. and Khasminski, R. Z. (1981). Statistical Estimation: Asymptotic Theory. Springer, New York.
  • [16] Ibragimov, I. A. and Khas’minskij, R. Z. (1984). On the nonparametric estimation of a value of a linear functional in Gaussian white noise. Teor. Veroyatnost. i Primenen. 29 19–32. (Russian. English summary.)
  • [17] Klemela, J. and Tsybakov, A. B. (2001). Sharp adaptive estimation of linear functionals. Ann. Statist. 29 1567–1600.
  • [18] Korostelev, A. and Tsybakov, A. (1993). Minimax Theory of Image Reconstruction. Lecture Notes in Statistics 82. Springer, New York.
  • [19] Lepskii, O. (1990). On a problem of adaptive estimation in Gaussian white noise. Teor. Veroyatnost. i Primenen. 35 454–466.
  • [20] Lepskii, O. (1991). Asymptotically minimax adaptive estimation I. Upper bounds. Optimally adaptive estimates. Teor. Veroyatnost. i Primenen. 36 682–697.
  • [21] Nemirovski, A. (2000). Topics in Nonparametric Statistics. In Ecole d’Eteé de Probabilités de Saint-Flour XXVII (M. Emery, A. Nemirovski, D. Voiculescu and P. Bernard, eds.). Lecture Notes in Mathematics 1738 87–285. Springer, New York.
  • [22] Pinsker, M. (1980). Optimal filtration of square-integrable signals in Gaussian noise. Problemy Peredachi Informatsii 16 120–133.
  • [23] Prakasa Rao, B. L. S. (1983). Nonparametric Functional Estimation. Academic Press, Orlando.
  • [24] Rosenblatt, M. (1991). Stochastic Curve Estimation. Institute of Mathematical Statistics, Hayward, CA.
  • [25] Simonoff, J. S. (1996). Smoothing Methods in Statistics. Springer, New York.
  • [26] Takezawa, K. (2005). Introduction to Nonparametric Regression. Wiley, Hoboken, NJ.
  • [27] Tsybakov, A. B. (2004). Introduction a l’Estimation Nonparamétrique. Springer, Berlin.
  • [28] Wasserman, L. (2006). All of Nonparametric Statistics. Springer, New York.