The Annals of Statistics

Focused information criterion and model averaging for generalized additive partial linear models

Xinyu Zhang and Hua Liang

Full-text: Open access


We study model selection and model averaging in generalized additive partial linear models (GAPLMs). Polynomial spline is used to approximate nonparametric functions. The corresponding estimators of the linear parameters are shown to be asymptotically normal. We then develop a focused information criterion (FIC) and a frequentist model average (FMA) estimator on the basis of the quasi-likelihood principle and examine theoretical properties of the FIC and FMA. The major advantages of the proposed procedures over the existing ones are their computational expediency and theoretical reliability. Simulation experiments have provided evidence of the superiority of the proposed procedures. The approach is further applied to a real-world data example.

Article information

Ann. Statist., Volume 39, Number 1 (2011), 174-200.

First available in Project Euclid: 3 December 2010

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression
Secondary: 62G20: Asymptotic properties 62G99: None of the above, but in this section

Additive models backfitting focus parameter generalized partially linear models marginal integration model average model selection polynomial spline shrinkage methods


Zhang, Xinyu; Liang, Hua. Focused information criterion and model averaging for generalized additive partial linear models. Ann. Statist. 39 (2011), no. 1, 174--200. doi:10.1214/10-AOS832.

Export citation


  • Akaike, H. (1973). Maximum likelihood identification of Gaussian autoregressive moving average models. Biometrika 22 203–217.
  • Bates, J. M. and Granger, C. M. J. (1969). The combination of forecasts. Operations Res. Quart. 20 451–468.
  • Buckland, S. T., Burnham, K. P. and Augustin, N. H. (1997). Model selection: An integral part of inference. Biometrics 53 603–618.
  • Buja, A., Hastie, T. and Tibshirani, R. (1989). Linear smoothers and additive models (with discussion). Ann. Statist. 17 453–555.
  • Burnham, K. P. and Anderson, D. R. (2002). Model Selection and Multimodel Inference: A Practical Information Theoretic Approach, 2nd ed. Springer, New York.
  • Carroll, R. J., Fan, J., Gijbels, I. and Wand, M. P. (1997). Generalized partially linear single-index models. J. Amer. Statist. Assoc. 92 477–489.
  • Claeskens, G. and Carroll, R. J. (2007). An asymptotic theory for model selection inference in general semiparametric problems. Biometrika 94 249–265.
  • Claeskens, G., Croux, C. and van Kerckhoven, J. (2006). Variable selection for logistic regression using a prediction-focused information criterion. Biometrics 62 972–979.
  • Claeskens, G., Croux, C. and van Kerckhoven, J. (2007). Prediction-focused model selection for autoregressive models. Aus. J. Statist. 49 359–379.
  • Claeskens, G. and Hjort, N. L. (2003). The focused information criterion (with discussion). J. Amer. Statist. Assoc. 98 900–916.
  • Claeskens, G. and Hjort, N. L. (2008). Model Selection and Model Averaging. Cambridge Univ. Press, Cambridge.
  • Danilov, D. and Magnus, J. R. (2004). On the harm that ignoring pretesting can cause. J. Econometrics 122 27–46.
  • de Boor, C. (2001). A Practical Guide to Splines. Springer, New York.
  • Draper, D. (1995). Assessment and propagation of model uncertainty. J. R. Stat. Soc. Ser. B 57 45–70.
  • Fan, J., Feng, Y. and Song, R. (2009). Nonparametric independence screening in sparse ultra-high dimensional additive models. Technical report, Dept. Operations Research and Financial Engineering, Princeton Univ.
  • Hand, D. J. and Vinciotti, V. (2003). Local versus global models for classification problems: Fitting models where it matters. Amer. Statist. 57 124–131.
  • Hansen, B. E. (2005). Challenges for econometric model selection. Econ. Theory 21 60–68.
  • Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. Chapman & Hall, London.
  • Härdle, W., Huet, S., Mammen, E. and Sperlich, S. (2004a). Bootstrap inference in semiparametric generalized additive models. Econom. Theory 20 265–300.
  • Härdle, W., Müller, M., Sperlich, S. and Werwatz, A. (2004b). Nonparametric and Semiparametric Models. Springer, New York.
  • Hjort, N. L. and Claeskens, G. (2003). Frequentist model average estimators (with discussion). J. Amer. Statist. Assoc. 98 879–945.
  • Hjort, N. L. and Claeskens, G. (2006). Focussed information criteria and model averaging for Cox’s hazard regression model. J. Amer. Statist. Assoc. 101 1449–1464.
  • Huang, J. (1998). Functional ANOVA models for generalized regression. J. Multivariate Anal. 67 49–71.
  • Huang, J. (1999). Efficient estimation of the partly linear additive Cox model. Ann. Statist. 27 1536–1563.
  • Hunsberger, S. (1994). Semiparametric regression in likelihood-based models. J. Amer. Statist. Assoc. 89 1354–1365.
  • Hunsberger, S., Albert, P., Follmann, D. and Suh, E. (2002). Parametric and semiparametric approaches to testing for seasonal trend in serial count data. Biostatistics 3 289–298.
  • Kim, T. H. and White, H. (2001). James–Stein-type estimators in large samples with application to the least absolute deviations estimator. J. Amer. Statist. Assoc. 96 697–705.
  • Leeb, H. and Pötscher, B. M. (2006). Can one estimate the conditional distribution of post-model-selection estimators? Ann. Statist. 34 2554–2591.
  • Leung, G. and Barron, A. R. (2006). Information theory and mixing least-squares regressions. IEEE Trans. Inform. Theory 52 3396–3410.
  • Liang, H. (2008). Generalized partially linear models with missing covariates. J. Multivariate Anal. 99 880–895.
  • Liang, H., Thurston, S., Ruppert, D., Apanasovich, T. and Hauser, R. (2008). Additive partial linear models with measurement errors. Biometrika 95 667–678.
  • Lin, X. H. and Carroll, R. J. (2001). Semiparametric regression for clustered data using generalized estimating equations. J. Amer. Statist. Assoc. 96 1045–1056.
  • Linton, O. B. and Nielsen, J. P. (1995). A kernel method of estimating structured nonparametric regression based on marginal integration. Biometrika 82 93–101.
  • Müller, M. and Rönz, B. (2000). Credit scoring using semiparametric methods. In Measuring Risk in Complex Stochastic Systems. Lecture Notes in Statistics (J. Franks, W. Härdle and G. Stahl, eds.) 147 83–98. Springer, Berlin.
  • Pollard, D. (1991). Asymptotics for least absolute deviation regression estimators. Econom. Theory 7 186–199.
  • Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461–464.
  • Severini, T. A. and Staniswalis, J. G. (1994). Quasi-likelihood estimation in semiparametric models. J. Amer. Statist. Assoc. 89 501–511.
  • Shen, X., Huang, H. C. and Ye, J. (2004). Inference after model selection. J. Amer. Statist. Assoc. 99 751–762.
  • Shiboski, C. S. (1998). Generalized additive models for current status data. Lifetime Data Anal. 4 29–50.
  • Stone, C. J. (1986). The dimensionality reduction principle for generalized additive models. Ann. Statist. 14 590–606.
  • Stone, C. J. (1994). The use of polynomial spline and their tensor products in multivariate function estimation (with discussion). Ann. Statist. 22 118–184.
  • Stone, C. J., Hansen, M. H., Kooperberg, C. and Truong, Y. K. (1997). Polynomial splines and their tensor products in extended linear modeling (with discussion). Ann. Statist. 25 1371–1470.
  • Sun, J., Kopciuk, A. K. and Lu, X. (2008). Polynomial spline estimation of partially linear single-index proportional hazards regression models. Comput. Statist. Data Anal. 53 176–188.
  • van der Vaart, A. and Wellner, J. (1996). Weak Convergence and Empirical Processes. Springer, New York.
  • Xue, L. and Yang, L. (2006). Additive coefficient modeling via polynomial spline. Statist. Sinica 16 1423–1446.
  • YANG, Y. (2001). Adaptive regression by mixing. J. Amer. Statist. Assoc. 96 574–586.
  • Yu, K., Park, B. U. and Mammen, E. (2008). Smooth backfitting in generalized additive models. Ann. Statist. 36 228–260.