## Electronic Journal of Statistics

### Ordered smoothers with exponential weighting

#### Abstract

The main goal in this paper is to propose a new approach to deriving oracle inequalities related to the exponential weighting method. The paper focuses on recovering an unknown vector from noisy data with the help of the family of ordered smoothers [12]. The estimators withing this family are aggregated using the exponential weighting method and the aim is to control the risk of the aggregated estimate. Based on the natural probabilistic properties of the unbiased risk estimate, we derive new oracle inequalities for the mean square risk and show that the exponential weighting permits to improve Kneip’s oracle inequality.

#### Article information

Source
Electron. J. Statist., Volume 7 (2013), 2395-2419.

Dates
First available in Project Euclid: 30 September 2013

https://projecteuclid.org/euclid.ejs/1380546591

Digital Object Identifier
doi:10.1214/13-EJS849

Mathematical Reviews number (MathSciNet)
MR3108818

Zentralblatt MATH identifier
1349.62129

#### Citation

Chernousova, Elena; Golubev, Yuri; Krymova, Ekaterina. Ordered smoothers with exponential weighting. Electron. J. Statist. 7 (2013), 2395--2419. doi:10.1214/13-EJS849. https://projecteuclid.org/euclid.ejs/1380546591

#### References

• [1] Alquier, P. and Lounici, K. (2011). Pac-bayesian theorems for sparse regression estimation with exponential weights., Electronic Journal of Statistics 5 127–145.
• [2] Arias-Castro, E. and Lounici, K. Variable Selection with Exponential Weights and $\ell_0$-Penalization. arXiv, :1208.2635.
• [3] Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle., Proc. 2nd Intern. Symp. Inf. Theory 267–281.
• [4] Catoni, O. (2004)., Statistical Learning Theory and Stochastic Optimization. Lectures Notes in Math. 1851. Springer-Verlag, Berlin.
• [5] Dalayan, A. and Salmon, J. (2012). Sharp oracle inequalities for aggregation of affine estimators., Ann. Statist. 40 2327–2355.
• [6] Demmler, A. and Reinsch, C. (1975). Oscillation matrices with spline smoothing., Numerische Mathematik 24 375–382.
• [7] Engl, H. W., Hanke, M., and Neubauer, A. (1996)., Regularization of Inverse Problems. Mathematics and its Applications 375. Kluwer Academic Publishers Group, Dordrecht.
• [8] Golubev, Yu. (2010). On universal oracle inequalities related to high dimensional linear models., Ann. Statist. 38 2751–2780.
• [9] Golubev, G. (2012). Exponential weighting and oracle inequalities for projection estimates., Problems of Information Transmission 48 269–280.
• [10] Green, P. J. and Silverman, B. W. (1994)., Nonparametric Regression and Generalized Linear Models. A Roughness Penalty Approach. Chapman and Hall.
• [11] Juditsky, A. and Nemirovski, A. (2000). Functional aggregation for nonparametric regression., Ann. Statist. 28 681–712.
• [12] Kneip, A. (1994). Ordered linear smoothers., Ann. Statist. 22 835–866.
• [13] Lecué, G. (2007). Simultaneous adaptation to the margin and to complexity in classification., Ann. Statist. 35 1698–1721.
• [14] Leung, G. and Barron, A. (2006). Information theory and mixing least-squares regressions., IEEE Transactions on Information Theory 52 3396–3410.
• [15] Mallows, C. L. (1973). Some comments on $C_p$., Technometrics 15 661–675.
• [16] Nemirovski, A. (2000)., Topics in Non-Parametric Statistics. Lectures Notes in Math. 1738. Springer-Verlag, Berlin.
• [17] Nussbaum, M. (1985). Spline smoothing in regression models and asymptotic efficiency in $L_2$., Ann. Statist. 13 984–997.
• [18] Rigollet, Ph. and Tsybakov, A. (2007). Linear and convex aggregation of density estimators., Math. Methods Statist. 16 260–280.
• [19] Rigollet, Ph. and Tsybakov, A. (2012). Sparse estimation by exponential weighting., Statist. Sci. 27 558–575.
• [20] Speckman, P. (1985). Spline smoothing and optimal rates of convergence in nonparametric regression., Ann. Statist. 13 970–983.
• [21] Stein, C. (1981). Estimation of the mean of a multivariate normal distribution., Ann. Statist. 9 1135–1151.
• [22] Tikhonov, A. N. and Arsenin, V. A. (1977)., Solution of Ill-posed Problems. Translated from the Russian. Preface by translation editor Fritz John. Scripta Series in Mathematics. V. H. Winston & Sons, Washington, D.C.: John Wiley & Sons, New York.
• [23] Wahba, G. (1990)., Spline Models for Observational Data. SIAM, Philadelphia.
• [24] Yang, Y. (2000). Combining different procedures for adaptive regression., J. Multivariate Anal. 74 135–161.