Electronic Journal of Statistics

Penalized profiled semiparametric estimating functions

Lan Wang, Bo Kai, Cédric Heuchenne, and Chih-Ling Tsai

Full-text: Open access

Abstract

In this paper, we propose a general class of penalized profiled semiparametric estimating functions which is applicable to a wide range of statistical models, including quantile regression, survival analysis, and missing data, among others. It is noteworthy that the estimating function can be non-smooth in the parametric and/or nonparametric components. Without imposing a specific functional structure on the nonparametric component or assuming a conditional distribution of the response variable for the given covariates, we establish a unified theory which demonstrates that the resulting estimator for the parametric component possesses the oracle property. Monte Carlo studies indicate that the proposed estimator performs well. An empirical example is also presented to illustrate the usefulness of the new method.

Article information

Source
Electron. J. Statist. Volume 7 (2013), 2656-2682.

Dates
First available in Project Euclid: 30 October 2013

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1383138545

Digital Object Identifier
doi:10.1214/13-EJS859

Mathematical Reviews number (MathSciNet)
MR3138833

Zentralblatt MATH identifier
1274.62261

Subjects
Primary: 62G05: Estimation 62G08: Nonparametric regression
Secondary: 62G20: Asymptotic properties

Keywords
Profiled semiparametric estimating functions nonconvex penalty non-smooth estimating functions

Citation

Wang, Lan; Kai, Bo; Heuchenne, Cédric; Tsai, Chih-Ling. Penalized profiled semiparametric estimating functions. Electron. J. Statist. 7 (2013), 2656--2682. doi:10.1214/13-EJS859. https://projecteuclid.org/euclid.ejs/1383138545.


Export citation

References

  • Bickel, P. J. and Li, B. (2006). Regularization in statistics (with discussion)., Test 15 271–344.
  • Bunea, F. (2004). Consistent covariate selection and post model selection inference in semiparametric regression., The Annals of Statistics 32 898–927.
  • Carroll, R. J., Fan, J., Gijbels, I. and Wand, M. P. (1997). Generalized partially linear single-index models., Journal of the American Statistical Association 92 477–489.
  • Chen, X., Linton, O. B. and Keilegom, I. V. (2003). Estimation of semiparametric models when the criterion function is not smooth., Econometrica 71 1591–1608.
  • Fan, J., Feng, Y. and Song, R. (2011). Nonparametric independence screening in sparse ultra-high dimensional additive models., Journal of the American Statistical Association 106 544–557.
  • Fan, J., Härdle, W. and Mammen, E. (1998). Direct estimation of low-dimensional components in additive models., The Annals of Statistics 26 943–971.
  • Fan, J. and Huang, T. (2005). Profile likelihood inferences on semiparametric varying-coefficient partially linear models., Bernoulli 11 1031–1057.
  • Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties., Journal of the American Statistical Association 96 1348–1360.
  • Fan, J. and Lv, J. (2008). Sure independence screening for ultrahigh dimensional feature space (with discussion)., Journal of the Royal Statistical Society: Series B (Statistical Methodology) 70 849–911.
  • Fan, J. and Yao, Q. (2003)., Nonlinear Time Series: Nonparametric and Parametric Methods. Springer-Verlag, New York.
  • Fu, W. J. (2003). Penalized estimating equations., Biometrics 59 126–132.
  • Greenshtein, E. and Ritov, Y. (2004). Persistence in high-dimensional linear predictor selection and the virtue of overparametrization., Bernoulli 10 971–988.
  • Hunter, D. R. and Lange, K. (2000). Quantile regression via an MM algorithm., Journal of Computational and Graphical Statistics 9 60–77.
  • Hunter, D. R. and Lange, K. (2004). A tutorial on MM algorithms., The American Statistician 58 30–37.
  • Hunter, D. R. and Li, R. (2005). Variable selection using MM algorithms., The Annals of Statistics 33 1617–1642.
  • Ichimura, H. (1993). Semiparametric least squares (SLS) and weighted SLS estimation of single-index models., Journal of Econometrics 58 71–120.
  • Johnson, B. A., Lin, D. Y. and Zeng, D. (2008). Penalized estimating functions and variable selection in semiparametric regression models., Journal of the American Statistical Association 103 672–680.
  • Kai, B., Li, R. and Zou, H. (2011). New efficient estimation and variable selection methods for semiparametric varying-coefficient partially linear models., The Annals of Statistics 39 305–332.
  • Lee, S. (2003). Efficient semiparametric estimation of a partially linear quantile regression model., Econometric Theory 19 1–31.
  • Li, R. and Liang, H. (2008). Variable selection in semiparametric regression modeling., The Annals of Statistics 36 261–286.
  • Liang, H. and Li, R. (2009). Variable selection for partially linear models with measurement errors., Journal of the American Statistical Association 104 234–248.
  • Liang, H., Wang, H. and Tsai, C.-L. (2012). Profiled forward regression for ultrahigh dimensional variable screening in semiparametric partially linear models., Statistica Sinica 22 531–554.
  • Liang, H., Wang, S., Robins, J. M. and Carroll, R. J. (2004). Estimation in partially linear models with missing covariates., Journal of the American Statistical Association 99 357–367.
  • Liang, H., Liu, X., Li, R. and Tsai, C.-L. (2010). Estimation and testing for partially linear single-index models., The Annals of Statistics 38 3811–3836.
  • Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the Lasso., The Annals of Statistics 34 1436–1462.
  • Ortega, J. M. and Rheinboldt, W. C. (1970)., Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, San Diego.
  • Pagan, A. and Ullah, A. (1999)., Nonparametric Econometrics. Cambridge University Press, New York.
  • Schwarz, G. (1978). Estimating the dimension of a model., The Annals of Statistics 6 461–464.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso., Journal of the Royal Statistical Society, Series B (Methodological) 58 267–288.
  • Tsay, R. S. (2005)., Analysis of Financial Time Series, 2nd Edition. Wiley-Interscience, New York.
  • van der Vaart, A. W. and Wellner, J. A. (1996)., Weak Convergence and Empirical Processes: With Applications to Statistics. Springer-Verlag, New York.
  • Wang, H. (2009). Forward regression for ultra-high dimensional variable screening., Journal of the American Statistical Association 104 1512–1524.
  • Wang, H., Li, R. and Tsai, C.-L. (2007). Tuning parameter selectors for the smoothly clipped absolute deviation method., Biometrika 94 553–568.
  • Wang, H. J. and Wang, L. (2009). Locally weighted censored quantile regression., Journal of the American Statistical Association 104 1117–1128.
  • Wang, H. and Xia, Y. (2009). Shrinkage estimation of the varying coefficient model., Journal of the American Statistical Association 104 747–757.
  • Wang, C.-Y., Wang, S., Gutierrez, R. G. and Carroll, R. J. (1998). Local linear regression for generalized linear models with missing data., The Annals of Statistics 26 1028–1050.
  • Wang, L., Liu, X., Liang, H. and Carroll, R. J. (2011). Estimation and variable selection for generalized additive partial linear models., The Annals of Statistics 39 1827–1851.
  • Xie, H. and Huang, J. (2009). SCAD-penalized regression in high-dimensional partially linear models., The Annals of Statistics 37 673–696.
  • Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty., The Annals of Statistics 38 894–942.
  • Zhao, P. and Yu, B. (2006). On model selection consistency of Lasso., Journal of Machine Learning Research 7 2541–2563.