The Annals of Statistics

Estimation and variable selection for generalized additive partial linear models

Li Wang, Xiang Liu, Hua Liang, and Raymond J. Carroll

Full-text: Open access

Abstract

We study generalized additive partial linear models, proposing the use of polynomial spline smoothing for estimation of nonparametric functions, and deriving quasi-likelihood based estimators for the linear parameters. We establish asymptotic normality for the estimators of the parametric components. The procedure avoids solving large systems of equations as in kernel-based procedures and thus results in gains in computational simplicity. We further develop a class of variable selection procedures for the linear parameters by employing a nonconcave penalized quasi-likelihood, which is shown to have an asymptotic oracle property. Monte Carlo simulations and an empirical example are presented for illustration.

Article information

Source
Ann. Statist., Volume 39, Number 4 (2011), 1827-1851.

Dates
First available in Project Euclid: 26 July 2011

Permanent link to this document
https://projecteuclid.org/euclid.aos/1311688537

Digital Object Identifier
doi:10.1214/11-AOS885

Mathematical Reviews number (MathSciNet)
MR2893854

Zentralblatt MATH identifier
1227.62053

Subjects
Primary: 62G08: Nonparametric regression
Secondary: 62G20: Asymptotic properties 62G99: None of the above, but in this section

Keywords
Backfitting generalized additive models generalized partially linear models LASSO nonconcave penalized likelihood penalty-based variable selection polynomial spline quasi-likelihood SCAD shrinkage methods

Citation

Wang, Li; Liu, Xiang; Liang, Hua; Carroll, Raymond J. Estimation and variable selection for generalized additive partial linear models. Ann. Statist. 39 (2011), no. 4, 1827--1851. doi:10.1214/11-AOS885. https://projecteuclid.org/euclid.aos/1311688537


Export citation

References

  • Breiman, L. (1996). Heuristics of instability and stabilization in model selection. Ann. Statist. 24 2350–2383.
  • Buja, A., Hastie, T. and Tibshirani, R. (1989). Linear smoothers and additive models (with discussion). Ann. Statist. 17 453–555.
  • Carroll, R. J., Fan, J. Q., Gijbels, I. and Wand, M. P. (1997). Generalized partially linear single-index models. J. Amer. Statist. Assoc. 92 477–489.
  • Carroll, R. J., Maity, A., Mammen, E. and Yu, K. (2009). Nonparametric additive regression for repeatedly measured data. Biometrika 96 383–398.
  • Craven, P. and Wahba, G. (1979). Smoothing noisy data with spline functions. Numer. Math. 31 377–403.
  • de Boor, C. (2001). A Practical Guide to Splines, revised ed. Applied Mathematical Sciences 27. Springer, New York.
  • Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360.
  • Fan, J. and Li, R. (2006). Statistical challenges with high dimensionality: Feature selection in knowledge discovery. In International Congress of Mathematicians. Vol. III 595–622. Eur. Math. Soc., Zürich.
  • Frank, I. E. and Friedman, J. H. (1993). A statistical view of some chemometrics regression tools (with discussion). Technometrics 35 109–148.
  • Härdle, W., Huet, S., Mammen, E. and Sperlich, S. (2004). Bootstrap inference in semiparametric generalized additive models. Econometric Theory 20 265–300.
  • Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. Monographs on Statistics and Applied Probability 43. Chapman and Hall, London.
  • Huang, J. (1998). Functional ANOVA models for generalized regression. J. Multivariate Anal. 67 49–71.
  • Huang, J. (1999). Efficient estimation of the partially linear additive Cox model. Ann. Statist. 27 1536–1563.
  • Hunter, D. and Li, R. (2005). Variable selection using MM algorithms. Ann. Statist. 33 1617–1642.
  • Li, R. and Liang, H. (2008). Variable selection in semiparametric regression modeling. Ann. Statist. 36 261–286.
  • Li, Y. and Ruppert, D. (2008). On the asymptotics of penalized splines. Biometrika 95 415–436.
  • Lin, X. and Carroll, R. J. (2006). Semiparametric estimation in general repeated measures problems. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 69–88.
  • Linton, O. and Nielsen, J. P. (1995). A kernel method of estimating structured nonparametric regression based on marginal integration. Biometrika 82 93–101.
  • Marx, B. D. and Eilers, P. H. C. (1998). Direct generalized additive modeling with penalized likelihood. Comput. Statist. Data Anal. 28 193–209.
  • McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd ed. Monographs on Statistics and Applied Probability 37. Chapman and Hall, London.
  • Nelder, J. A. and Wedderburn, R. W. M. (1972). Generalized linear models. J. Roy. Statist. Soc. Ser. A 135 370–384.
  • Ruppert, D., Wand, M. and Carroll, R. (2003). Semiparametric Regression. Cambridge Univ. Press, Cambridge.
  • Severini, T. A. and Staniswalis, J. G. (1994). Quasi-likelihood estimation in semiparametric models. J. Amer. Statist. Assoc. 89 501–511.
  • Smith, J. W., Everhart, J. E., Dickson, W. C., Knowler, W. C. and Johannes, R. S. (1988). Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In Proc. Annu. Symp. Comput. Appl. Med. Care 261–265. IEEE Computer Society Press, Washington, DC.
  • Stone, C. J. (1985). Additive regression and other nonparametric models. Ann. Statist. 13 689–705.
  • Stone, C. J. (1986). The dimensionality reduction principle for generalized additive models. Ann. Statist. 14 590–606.
  • Stone, C. J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation. Ann. Statist. 22 118–184.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58 267–288.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes with Applications to Statistics. Springer, New York.
  • Wood, S. N. (2004). Stable and efficient multiple smoothing parameter estimation for generalized additive models. J. Amer. Statist. Assoc. 99 673–686.
  • Wood, S. N. (2006). Generalized Additive Models. Chapman & Hall/CRC Press, Boca Raton, FL.
  • Xue, L. and Yang, L. (2006). Additive coefficient modeling via polynomial spline. Statist. Sinica 16 1423–1446.
  • Yu, K. and Lee, Y. K. (2010). Efficient semiparametric estimation in generalized partially linear additive models. J. Korean Statist. Soc. 39 299–304.
  • Yu, K., Park, B. U. and Mammen, E. (2008). Smooth backfitting in generalized additive models. Ann. Statist. 36 228–260.
  • Zou, H. (2006). The adaptive Lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418–1429.

Supplemental materials

  • Supplementary material: Detailed proofs and additional simulation results of: Estimation and variable selection for generalized additive partial linear models. The supplemental materials contain detailed proofs and additional simulation results.