The Annals of Statistics

Parameter tuning in pointwise adaptation using a propagation approach

Vladimir Spokoiny and Céline Vial

Full-text: Open access

Enhanced PDF (303 KB)

Abstract

This paper discusses the problem of adaptive estimation of a univariate object like the value of a regression function at a given point or a linear functional in a linear inverse problem. We consider an adaptive procedure originated from Lepski [Theory Probab. Appl. 35 (1990) 454–466.] that selects in a data-driven way one estimate out of a given class of estimates ordered by their variability. A serious problem with using this and similar procedures is the choice of some tuning parameters like thresholds. Numerical results show that the theoretically recommended proposals appear to be too conservative and lead to a strong oversmoothing effect. A careful choice of the parameters of the procedure is extremely important for getting the reasonable quality of estimation. The main contribution of this paper is the new approach for choosing the parameters of the procedure by providing the prescribed behavior of the resulting estimate in the simple parametric situation. We establish a non-asymptotical “oracle” bound, which shows that the estimation risk is, up to a logarithmic multiplier, equal to the risk of the “oracle” estimate that is optimally selected from the given family. A numerical study demonstrates a good performance of the resulting procedure in a number of simulated examples.

Article information

Source
Ann. Statist., Volume 37, Number 5B (2009), 2783-2807.

Dates
First available in Project Euclid: 17 July 2009

Permanent link to this document
https://projecteuclid.org/euclid.aos/1247836669

Digital Object Identifier
doi:10.1214/08-AOS607

Mathematical Reviews number (MathSciNet)
MR2541447

Zentralblatt MATH identifier
1173.62028

Subjects
Primary: 62G05: Estimation 62G05: Estimation
Secondary: 62G10: Hypothesis testing 62G10: Hypothesis testing

Keywords
Linear functional inverse problem propagation oracle

Citation

Spokoiny, Vladimir; Vial, Céline. Parameter tuning in pointwise adaptation using a propagation approach. Ann. Statist. 37 (2009), no. 5B, 2783--2807. doi:10.1214/08-AOS607. https://projecteuclid.org/euclid.aos/1247836669


Export citation

References

  • Bauer, F. (2007). Some considerations concerning regularization and parameter choice algorithms. Inverse Problems 23 837–858.
    Mathematical Reviews (MathSciNet): MR2309678
    Digital Object Identifier: doi:10.1088/0266-5611/23/2/021
    Zentralblatt MATH: 1109.62020
  • Birgé, L. and Massart, P. (1993). Rate of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113–150.
    Mathematical Reviews (MathSciNet): MR1240719
    Zentralblatt MATH: 0805.62037
    Digital Object Identifier: doi:10.1007/BF01199316
  • Birgé, L. and Massart, P. (1998). Minimum contrast estimators on sieves: Exponential bounds and rates of convergence. Bernoulli 4 329–375.
    Mathematical Reviews (MathSciNet): MR1653272
    Digital Object Identifier: doi:10.2307/3318720
    Project Euclid: euclid.bj/1174324984
    Zentralblatt MATH: 0954.62033
  • Birgé, L. (2006). Model selection via testing: an alternative to (penalized) maximum likelihood estimators. Ann. Inst. H. Poincaré Probab. Statist. 42 273–325.
    Mathematical Reviews (MathSciNet): MR2219712
    Zentralblatt MATH: 05024238
    Digital Object Identifier: doi:10.1016/j.anihpb.2005.04.004
  • Cai, T. T. and Hall, P. (2006). Prediction in functional linear regression Ann. Statist. 34 2159–2179.
    Mathematical Reviews (MathSciNet): MR2291496
    Zentralblatt MATH: 1106.62036
    Digital Object Identifier: doi:10.1214/009053606000000830
    Project Euclid: euclid.aos/1169571793
  • Cavalier, L. (2001). On the problem of local adaptive estimation in tomography. Bernoulli 7 63–78.
    Mathematical Reviews (MathSciNet): MR1811744
    Digital Object Identifier: doi:10.2307/3318602
    Project Euclid: euclid.bj/1080572339
    Zentralblatt MATH: 0966.62021
  • Goldenshluger, A. (1998). Nonparametric estimation of transfer functions: Rates of convergence and adaptation. IEEE Trans. Inform. Theory 44 644–658.
    Mathematical Reviews (MathSciNet): MR1607738
    Digital Object Identifier: doi:10.1109/18.661510
    Zentralblatt MATH: 0902.62043
  • Goldenshluger, A. (1999). On pointwise adaptive nonparametric deconvolution. Bernoulli 5 907–925.
    Mathematical Reviews (MathSciNet): MR1715444
    Digital Object Identifier: doi:10.2307/3318449
    Project Euclid: euclid.bj/1171290404
    Zentralblatt MATH: 0953.62033
  • Goldenshluger, A. and Pereverzev, S. (2000). Adaptive estimation of linear functionals in Hilbert scales from indirect white noise observations. Probab. Theory Related Fields 118 169–186.
    Mathematical Reviews (MathSciNet): MR1790080
    Digital Object Identifier: doi:10.1007/s440-000-8013-3
    Zentralblatt MATH: 1055.62523
  • Goldenshluger, A. and Pereverzev, S. (2003). On adaptive inverse estimation of linear functionals in Hilbert scales. Bernoulli 9 783–807.
    Mathematical Reviews (MathSciNet): MR2047686
    Digital Object Identifier: doi:10.3150/bj/1066418878
    Project Euclid: euclid.bj/1066418878
    Zentralblatt MATH: 1055.62034
  • Golubev, G. K. (2004). The Method of Risk Envelope in Estimation of Linear Functionals. Problems Inform. Transmission 40 53–65. Translated from Problemy Peredachi Informatsii 1 (2004) 58–72.
    Mathematical Reviews (MathSciNet): MR2099020
  • Juditsky, A., Rigollet, P. and Tsybakov, A. B. (2008). Learning by mirror averaging. Ann. Statist. 36 2183–2206.
    Mathematical Reviews (MathSciNet): MR2458184
    Zentralblatt MATH: 05368488
    Digital Object Identifier: doi:10.1214/07-AOS546
    Project Euclid: euclid.aos/1223908089
  • Lepskii, O. V. (1990). A problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl. 35 454–466. Translated from Teor. Veroyatnost. i Primenen. 35 (1990) 459–470.
    Mathematical Reviews (MathSciNet): MR1091202
    Zentralblatt MATH: 0745.62083
  • Lepskii, O. V. (1991). Asymptotically minimax adaptive estimation. I. Upper Bounds. Optimally adaptive estimates. Theory Probab. Appl. 36 682–697.
    Mathematical Reviews (MathSciNet): MR1147167
    Zentralblatt MATH: 0776.62039
  • Lepskii, O. V. (1992). Asymptotically minimax adaptive estimation. II. Schemes without optimal adaptation: Adaptive estimators. Theory Probab. Appl. 37 433–448.
    Mathematical Reviews (MathSciNet): MR1214353
  • Lepski, O. V., Mammen, E. and Spokoiny, V. G. (1997). Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors. Ann. Statist. 25 929–947.
    Mathematical Reviews (MathSciNet): MR1447734
    Zentralblatt MATH: 0885.62044
    Digital Object Identifier: doi:10.1214/aos/1069362731
    Project Euclid: euclid.aos/1069362731
  • Lepski, O. V. and Spokoiny, V. (1997). Optimal pointwise adaptive methods in nonparametric estimation. Ann. Statist. 25 2512–2546.
    Mathematical Reviews (MathSciNet): MR1604408
    Zentralblatt MATH: 0894.62041
    Digital Object Identifier: doi:10.1214/aos/1030741083
    Project Euclid: euclid.aos/1030741083
  • Spokoiny, V. (2009). Local Parametric Estimation. Springer, New York. To appear.
    Zentralblatt MATH: 1160.62084
    Digital Object Identifier: doi:10.1214/08-AOS612
    Project Euclid: euclid.aos/1239369026

The Institute of Mathematical Statistics

The Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more