The Annals of Statistics

The EFM approach for single-index models

Xia Cui, Wolfgang Karl Härdle, and Lixing Zhu

Full-text: Open access

Abstract

Single-index models are natural extensions of linear models and circumvent the so-called curse of dimensionality. They are becoming increasingly popular in many scientific fields including biostatistics, medicine, economics and financial econometrics. Estimating and testing the model index coefficients β is one of the most important objectives in the statistical analysis. However, the commonly used assumption on the index coefficients, ‖β‖ = 1, represents a nonregular problem: the true index is on the boundary of the unit ball. In this paper we introduce the EFM approach, a method of estimating functions, to study the single-index model. The procedure is to first relax the equality constraint to one with (d − 1) components of β lying in an open unit ball, and then to construct the associated (d − 1) estimating functions by projecting the score function to the linear space spanned by the residuals with the unknown link being estimated by kernel estimating functions. The root-n consistency and asymptotic normality for the estimator obtained from solving the resulting estimating equations are achieved, and a Wilks type theorem for testing the index is demonstrated. A noticeable result we obtain is that our estimator for β has smaller or equal limiting variance than the estimator of Carroll et al. [J. Amer. Statist. Assoc. 92 (1997) 447–489]. A fixed-point iterative scheme for computing this estimator is proposed. This algorithm only involves one-dimensional nonparametric smoothers, thereby avoiding the data sparsity problem caused by high model dimensionality. Numerical studies based on simulation and on applications suggest that this new estimating system is quite powerful and easy to implement.

Article information

Source
Ann. Statist., Volume 39, Number 3 (2011), 1658-1688.

Dates
First available in Project Euclid: 25 July 2011

Permanent link to this document
https://projecteuclid.org/euclid.aos/1311600279

Digital Object Identifier
doi:10.1214/10-AOS871

Mathematical Reviews number (MathSciNet)
MR2850216

Zentralblatt MATH identifier
1221.62062

Subjects
Primary: 62G08: Nonparametric regression 62G08: Nonparametric regression 62G20: Asymptotic properties

Keywords
Single-index models index coefficients estimating equations asymptotic properties iteration

Citation

Cui, Xia; Härdle, Wolfgang Karl; Zhu, Lixing. The EFM approach for single-index models. Ann. Statist. 39 (2011), no. 3, 1658--1688. doi:10.1214/10-AOS871. https://projecteuclid.org/euclid.aos/1311600279


Export citation

References

  • Andrews, D. W. K. (1987). Conssitency in nonlinear econometric models: A genetic uniform law of large numbers. Econometrica 55 1465–1471.
  • Carroll, R. J., Ruppert, D. and Welsh, A. H. (1998). Local estimating equations. J. Amer. Statist. Assoc. 93 214–227.
  • Carroll, R. J., Fan, J., Gijbels, I. and Wand, M. P. (1997). Generalized partially linear single-index models. J. Amer. Statist. Assoc. 92 447–489.
  • Chang, Z. Q., Xue, L. G. and Zhu, L. X. (2010). On an asymptotically more efficient estimation of the single-index model. J. Multivariate Anal. 101 1898–1901.
  • Cui, X., Härdle, W. and Zhu, L. (2010). Supplementary materials for “The EFM approach for single-index models.” DOI:10.1214/10-AOS871SUPP.
  • Fan, J. and Gijbels, I. (1996). Local Polynomial Modeling and Its Applications. Chapman & Hall, London.
  • Fan, J., Heckman, N. E. and Wand, M. P. (1995). Local polynomial kernel regression for generalized linear models and quasi-likelihood functions. J. Amer. Statist. Assoc. 90 141–150.
  • Fan, J. and Jiang, J. (2007). Nonparametric inference with generalized likelihood ratio test. Test 16 409–478.
  • Härdle, W., Hall, P. and Ichimura, H. (1993). Optimal smoothing in single-index models. Ann. Statist. 21 157–178.
  • Härdle, W. and Mammen, E. (1993). Testing parametric versus nonparametric regression. Ann. Statist. 21 1926–1947.
  • Härdle, W., Mammen, E. and Müller, M. (1998). Testing parametric versus semiparametric modelling in generalized linear models. J. Amer. Statist. Assoc. 93 1461–1474.
  • Härdle, W., Mammen, E. and Proenca, I. (2001). A bootstrap test for single index models. Statistics 35 427–452.
  • Härdle, W. and Stoker, T. M. (1989). Investigating smooth multiple regression by method of average derivatives. J. Amer. Statist. Assoc. 84 986–995.
  • Heyde, C. C. (1997). Quasi-likelihood and Its Application: A General Approach to Optimal Parameter Estimation. Springer, New York.
  • Horowitz, J. L. and Härdle, W. (1996). Direct semiparametric estimation of a single-index model with discrete covariates. J. Amer. Statist. Assoc. 91 1632–1640.
  • Hristache, M., Juditski, A. and Spokoiny, V. (2001). Direct estimation of the index coefficients in a single-index model. Ann. Statist. 29 595–623.
  • Hristache, M., Juditsky, A., Polzehl, J. and Spokoiny, V. (2001). Structure adaptive approach for dimension reduction. Ann. Statist. 29 1537–1566.
  • Huh, J. and Park, B. U. (2002). Likelihood-based local polynomial fitting for single-index models. J. Multivariate Anal. 80 302–321.
  • Ichimura, H. (1993). Semiparametric least squares (SLS) and weighted SLS estimation of single-index models. J. Econometrics 58 71–120.
  • Kane, M., Holt, J. and Allen, B. (2004). Results concerning the generalized partially linear single-index model. J. Stat. Comput. Simul. 72 897–912.
  • Kohavi, R. (1996). Scaling up the accuracy of naive-Bayes classifiers: A decision-tree hybrid. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining 202–207. AAAI Press, Menlo Park, CA.
  • Kong, E., Linton, O. and Xia, Y. (2010). Uniform Bahadur representation for local polynomial estimates of M-regression and its application to the additive model. Econometric Theory 26 1529–1564.
  • Lin, W. and Kulasekera, K. B. (2007). Identifiability of single-index models and additive-index models. Biometrika 94 496–501.
  • Madalozzo, R. C. (2008). An analysis of income differentials by marital status. Estudos Econôicos 38 267–292.
  • McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd ed. Champman & Hall, London.
  • Murray, C. (1997). IQ and economic success. The Public Interest 128 21–35.
  • Polzehl, J. and Sperlich, S. (2009). A note on structural adaptive dimension reduction. J. Stat. Comput. Simul. 79 805–818.
  • Powell, J. L., Stock, J. H. and Stoker, T. M. (1989). Semiparametric estimation of index coefficients. Econometrica 57 1403–1430.
  • Wang, H. and Xia, Y. (2008). Sliced regression for dimension reduction. J. Amer. Statist. Assoc. 103 811–821.
  • Wang, J. L., Xue, L. G., Zhu, L. X. and Chong, Y. S. (2010). Estimation for a partial-linear single-index model. Ann. Statist. 38 246–274.
  • Xia, Y. (2006). Asymptotic distributions for two estimators of the single-index model. Econometric Theory 22 1112–1137.
  • Xia, Y., Tong, H., Li, W. K. and Zhu, L. (2002). An adaptive estimation of dimension reduction space (with discussions). J. R. Stat. Soc. Ser. B Stat. Methodol. 64 363–410.
  • Yu, Y. and Ruppert, D. (2002). Penalized spline estimation for partially linear single index models. J. Amer. Statist. Assoc. 97 1042–1054.
  • Zhou, J. and He, X. (2008). Dimension reduction based on constrained canonical correlation and variable filtering. Ann. Statist. 36 1649–1668.
  • Zhu, L. X. and Xue, L. G. (2006). Empirical likelihood confidence regions in a partially linear single-index model. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 549–570.
  • Zhu, L. P. and Zhu, L. X. (2009a). Nonconcave penalized inverse regression in single-index models with high dimensional predictors. J. Multivariate Anal. 100 862–875.
  • Zhu, L. P. and Zhu, L. X. (2009b). On distribution weighted partial least squares with diverging number of highly correlated predictors. J. R. Stat. Soc. Ser. B Stat. Methodol. 71 525–548.

Supplemental materials