We apply the results of Andresen et. al. (2014) on finite sample properties of sieve M-estimators and Andresen et. al. (2015) on the convergence of an alternating maximization procedure to analyse a sieve profile maximization estimator in the single index model with linear index function. The link function is approximated with $C^{3}$-Daubechies-wavelets with compact support. We derive results like Wilks phenomenon and Fisher Theorem in a finite sample setup even when the model is miss-specified. Furthermore we show that an alternating maximization procedure converges to the global maximizer and we assess the performance of Friedman’s projection pursuit procedure. The approach is based on showing that the conditions of Andresen et. al. (2014) and (2015) can be satisfied under a set of mild regularity and moment conditions on the link function, the regressors and the additive noise. The results allow to construct non-asymptotic confidence sets and to derive asymptotic bounds for the estimator as corollaries.
References
[2] A. Andresen and V. Spokoiny. Critical dimension in profile semiparametric estimation., Electron. J. Statist., 8(2) :3077–3125, 2014. MR3301302 06391376 10.1214/14-EJS982 euclid.ejs/1421330631
[2] A. Andresen and V. Spokoiny. Critical dimension in profile semiparametric estimation., Electron. J. Statist., 8(2) :3077–3125, 2014. MR3301302 06391376 10.1214/14-EJS982 euclid.ejs/1421330631
[4] A. Cohen, I. Daubechies, and P. Vial. Wavelets on the interval and fast wavelet transforms., Applied and computational harmonic analysis, 1:54–81, 1993. MR1256527 10.1006/acha.1993.1005[4] A. Cohen, I. Daubechies, and P. Vial. Wavelets on the interval and fast wavelet transforms., Applied and computational harmonic analysis, 1:54–81, 1993. MR1256527 10.1006/acha.1993.1005
[5] M. Delecroix, W. Haerdle, and M. Hristache. Efficient estimation in single-index regression. Technical report, SFB 373, Humboldt Univ. Berlin, 1997.[5] M. Delecroix, W. Haerdle, and M. Hristache. Efficient estimation in single-index regression. Technical report, SFB 373, Humboldt Univ. Berlin, 1997.
[6] R. M. Dudley. The sizes of compact subsets of hilbert space and continuity of gaussian processes., Journal of Functional Analysis, 1:290–330, 1967. MR220340 10.1016/0022-1236(67)90017-1[6] R. M. Dudley. The sizes of compact subsets of hilbert space and continuity of gaussian processes., Journal of Functional Analysis, 1:290–330, 1967. MR220340 10.1016/0022-1236(67)90017-1
[7] Jerome H. Friedman and Werner Stuetzle. Projection pursuit regression., Journal of the American Statistical Association, 76(376):817–823, 1981. MR650892 10.1080/01621459.1981.10477729[7] Jerome H. Friedman and Werner Stuetzle. Projection pursuit regression., Journal of the American Statistical Association, 76(376):817–823, 1981. MR650892 10.1080/01621459.1981.10477729
[8] W. Haerdle, P. Hall, and H. Ichimura. Optimal smoothing in single-index models., Ann. Statist., 21:157–178, 1993. MR1212171 0770.62049 10.1214/aos/1176349020 euclid.aos/1176349020
[8] W. Haerdle, P. Hall, and H. Ichimura. Optimal smoothing in single-index models., Ann. Statist., 21:157–178, 1993. MR1212171 0770.62049 10.1214/aos/1176349020 euclid.aos/1176349020
[9] Peter Hall. Estimating the direction in which a data set is most interesting., Probability Theory and Related Fields, 80:51–77, 1988. MR970471 0637.62037 10.1007/BF00348752[9] Peter Hall. Estimating the direction in which a data set is most interesting., Probability Theory and Related Fields, 80:51–77, 1988. MR970471 0637.62037 10.1007/BF00348752
[10] M. Hristache, A. Juditski, J. Polzehl, and V. Spokoiny. Structure adaptive approach for dimension reduction., Annals of Statistics, 29:595–623, 2001. MR1865333 1012.62043 10.1214/aos/1009210681[10] M. Hristache, A. Juditski, J. Polzehl, and V. Spokoiny. Structure adaptive approach for dimension reduction., Annals of Statistics, 29:595–623, 2001. MR1865333 1012.62043 10.1214/aos/1009210681
[11] Peter J. Huber. Projection pursuit., The Annals of Statistics, 13(2):435–475, 1985. MR790553 0595.62059 10.1214/aos/1176349519 euclid.aos/1176349519
[11] Peter J. Huber. Projection pursuit., The Annals of Statistics, 13(2):435–475, 1985. MR790553 0595.62059 10.1214/aos/1176349519 euclid.aos/1176349519
[12] H. Ichimura. Semiparametric least squares (sls) and weighted sls estimation of single-index models., J Econometrics, 58:71–120, 1993. MR1230981 0816.62079 10.1016/0304-4076(93)90114-K[12] H. Ichimura. Semiparametric least squares (sls) and weighted sls estimation of single-index models., J Econometrics, 58:71–120, 1993. MR1230981 0816.62079 10.1016/0304-4076(93)90114-K
[13] Lee K. Jones. On a conjecture of huber concerning the convergence of projection pursuit regression., Ann. Statist, 15(2):880–882, 1987. MR888447 0664.62061 10.1214/aos/1176350382 euclid.aos/1176350382
[13] Lee K. Jones. On a conjecture of huber concerning the convergence of projection pursuit regression., Ann. Statist, 15(2):880–882, 1987. MR888447 0664.62061 10.1214/aos/1176350382 euclid.aos/1176350382
[14] M.R. Kosorok., Introduction to Empirical Processes and Semiparametric Inference. Springer in Statistics, 2005. MR2724368 1180.62137[14] M.R. Kosorok., Introduction to Empirical Processes and Semiparametric Inference. Springer in Statistics, 2005. MR2724368 1180.62137
[15] S. Mendelson. Learning without concentration., arXiv :1401.0304, 2014. MR3367000 10.1145/2699439[15] S. Mendelson. Learning without concentration., arXiv :1401.0304, 2014. MR3367000 10.1145/2699439
[16] Whitney K Newey. Convergence rates and asymptotic normality for series estimators., Journal of Econometrics, 79(1):147–168, 1997. MR1457700 0873.62049 10.1016/S0304-4076(97)00011-0[16] Whitney K Newey. Convergence rates and asymptotic normality for series estimators., Journal of Econometrics, 79(1):147–168, 1997. MR1457700 0873.62049 10.1016/S0304-4076(97)00011-0
[17] Jammes L. Powell, James H. Stock, and Thomas M. Stoker. Semiparametric estimation of index coefficients., Econometrica, 57(6) :1403–1430, 1989. MR1035117 10.2307/1913713[17] Jammes L. Powell, James H. Stock, and Thomas M. Stoker. Semiparametric estimation of index coefficients., Econometrica, 57(6) :1403–1430, 1989. MR1035117 10.2307/1913713
[18] Xiaotong Shen. On methods of sieves and penalization., Ann. Statist., 25(6) :2555–2591, 1997. MR1604416 0895.62041 10.1214/aos/1030741085 euclid.aos/1030741085
[18] Xiaotong Shen. On methods of sieves and penalization., Ann. Statist., 25(6) :2555–2591, 1997. MR1604416 0895.62041 10.1214/aos/1030741085 euclid.aos/1030741085
[19] Vladimir Spokoiny. Parametric estimation. Finite sample theory., Ann. Statist., 40(6) :2877–2909, 2012. MR3097963 1296.62051 10.1214/12-AOS1054 euclid.aos/1360332187
[19] Vladimir Spokoiny. Parametric estimation. Finite sample theory., Ann. Statist., 40(6) :2877–2909, 2012. MR3097963 1296.62051 10.1214/12-AOS1054 euclid.aos/1360332187
[20] C. J. Stone. Optimal rates of convergence for nonparametric estimators., Ann. Statist., 8(6) :1348–1360, 1980. MR594650 0451.62033 10.1214/aos/1176345206 euclid.aos/1176345206
[20] C. J. Stone. Optimal rates of convergence for nonparametric estimators., Ann. Statist., 8(6) :1348–1360, 1980. MR594650 0451.62033 10.1214/aos/1176345206 euclid.aos/1176345206
[21] M. Talagrand. Majorizing measures: the generic chaining., Ann. Statist., 24(3) :1049–1103, 1996. MR1411488 0867.60017 10.1214/aop/1065725175 euclid.aop/1065725175
[21] M. Talagrand. Majorizing measures: the generic chaining., Ann. Statist., 24(3) :1049–1103, 1996. MR1411488 0867.60017 10.1214/aop/1065725175 euclid.aop/1065725175
[22] J. A. Tropp. User-friendly tail bounds for sums of random matrices., Foundations of Computational Mathematics, 12:389–434, 2012. MR2946459 10.1007/s10208-011-9099-z[22] J. A. Tropp. User-friendly tail bounds for sums of random matrices., Foundations of Computational Mathematics, 12:389–434, 2012. MR2946459 10.1007/s10208-011-9099-z
[23] Yingcun Xia. Asymptotic distributions for two estimators of the single-index model., Econometric Theory, 22 :1112–1137, 2006. MR2328530 1170.62323 10.1017/S0266466606060531[23] Yingcun Xia. Asymptotic distributions for two estimators of the single-index model., Econometric Theory, 22 :1112–1137, 2006. MR2328530 1170.62323 10.1017/S0266466606060531
[24] Yingcun Xia, H. Tong, W.K. Li, and L. Zhu. An adaptive estimation of dimension reduction space., Journal of the Royal Statistical Society, pages 363–410, 2002. MR1924297 1091.62028 10.1111/1467-9868.03411[24] Yingcun Xia, H. Tong, W.K. Li, and L. Zhu. An adaptive estimation of dimension reduction space., Journal of the Royal Statistical Society, pages 363–410, 2002. MR1924297 1091.62028 10.1111/1467-9868.03411
