Electronic Journal of Statistics

SiAM: A hybrid of single index models and additive models

Shujie Ma, Heng Lian, Hua Liang, and Raymond J. Carroll

Full-text: Open access

Abstract

While popular, single index models and additive models have potential limitations, a fact that leads us to propose SiAM, a novel hybrid combination of these two models. We first address model identifiability under general assumptions. The result is of independent interest. We then develop an estimation procedure by using splines to approximate unknown functions and establish the asymptotic properties of the resulting estimators. Furthermore, we suggest a two-step procedure for establishing confidence bands for the nonparametric additive functions. This procedure enables us to make global inferences. Numerical experiments indicate that SiAM works well with finite sample sizes, and are especially robust to model structures. That is, when the model reduces to either single-index or additive scenario, the estimation and inference results are comparable to those based on the true model, while when the model is misspecified, the superiority of our method can be very great.

Article information

Source
Electron. J. Statist., Volume 11, Number 1 (2017), 2397-2423.

Dates
Received: August 2016
First available in Project Euclid: 29 May 2017

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1496044838

Digital Object Identifier
doi:10.1214/17-EJS1291

Mathematical Reviews number (MathSciNet)
MR3656496

Zentralblatt MATH identifier
1364.62096

Subjects
Primary: 62G08: Nonparametric regression
Secondary: 62G20: Asymptotic properties 62J02: General nonlinear regression 62F12: Asymptotic properties of estimators

Keywords
Additive models global inference identifiability misspecification oracle estimator partially linear single index models regression spline simultaneous confidence band

Rights
Creative Commons Attribution 4.0 International License.

Citation

Ma, Shujie; Lian, Heng; Liang, Hua; Carroll, Raymond J. SiAM: A hybrid of single index models and additive models. Electron. J. Statist. 11 (2017), no. 1, 2397--2423. doi:10.1214/17-EJS1291. https://projecteuclid.org/euclid.ejs/1496044838


Export citation

References

  • [1] Bellman, R. E. (1961), Adaptive Control Processes, Princeton University Press, Princeton, N.J.
  • [2] Bosq, D. (1998), Nonparametric Statistic for Stochastic Process, Vol. 10 of Lecture notes in Statistics, 2 edn, Springer, New York.
  • [3] Breiman, L. and Friedman, J. H. (1985), ‘Estimating optimal transformations for multiple regression and correlations (with discussion)’, Journal of the American Statistical Association 80, 580–619.
  • [4] Buja, A., Hastie, T. and Tibshirani, R. (1989), ‘Linear smoothers and additive models’, The Annals of Statistics 17, 453–510.
  • [5] Carroll, R. J., Fan, J., Gijbels, I. and Wand, M. P. (1997), ‘Generalized partially linear single-index models’, Journal of the American Statistical Association 92(438), 477–489.
  • [6] Csörgö, M. and Révész, P. (1981), Strong Approximations in Probability and Statistics, Academic Press, New York-London.
  • [7] Cui, X., Härdle, W. and Zhu, L.-X. (2011), ‘The EFM approach for single-index models’, The Annals of Statistics 39, 1658–1688.
  • [8] de Boor, C. (2001), A Practical Guide to Splines, Vol. 27 of Applied Mathematical Sciences, revised edn, Springer-Verlag, New York.
  • [9] Demko, S. (1986), ‘Spectral bounds for $\left\vert a^-1\right\vert _\infty$’, Journal of Approximation Theory 48, 207–212.
  • [10] DeVore, R. A. and Lorentz, G. G. (1993), Constructive approximation, Vol. 303 of Grundlehren der Mathematischen Wissenschaften, Springer-Verlag, Berlin.
  • [11] Goldenshluger, A. and Lepski, O. (2009), ‘Structural adaptation via $L_p$-norm oracle inequalities’, Probability Theory and Related Fields 143(1-2), 41–71.
  • [12] Härdle, W., Hall, P. and Ichimura, H. (1993), ‘Optimal smoothing in single-index models’, The Annals of Statistics 21, 157–178.
  • [13] Härdle, W., Müller, M., Sperlich, S. and Werwatz, A. (2004), Nonparametric and Semiparametric Models, Springer-Verlag, New York.
  • [14] Hastie, T. and Tibshirani, R. (1990), Generalized Additive Models, Monographs on statistics and applied probability, 1st edn, Chapman and Hall, London; New York.
  • [15] Horowitz, J. L. (2009), Semiparametric and Nonparametric Methods in Econometrics, Springer, New York.
  • [16] Ichimura, H. (1993), ‘Semiparametric least squares (SLS) and weighted SLS estimation of single-index models’, Journal of Econometrics 58, 71–120.
  • [17] Liang, H., Thurston, S., Ruppert, D., Apanasovich, T. and Hauser, R. (2008), ‘Additive partial linear models with measurement errors’, Biometrika 95, 667–678.
  • [18] Lin, W. and Kulasekera, K. B. (2007), ‘Identifiability of single-index models and additive-index models’, Biometrika 94(2), 496–501.
  • [19] Liu, X., Wang, L. and Liang, H. (2011), ‘Estimation and variable selection for semiparametric additive partial linear models’, Statistica Sinica 21, 1225–1248.
  • [20] Lu, X. and Cheng, T. (2007), ‘Randomly censored partially linear single-index models’, Journal of Multivariate Analysis 98, 1895–1922.
  • [21] Ma, S. (2012), ‘Two-step spline estimating equations for generalized additive partially linear models with large cluster sizes’, The Annals of Statistics 40, 2943–2972.
  • [22] Ma, S. and Yang, L. (2011), ‘A jump-detecting procedure based on polynomial spline estimation’, Journal of Nonparametric Statistics 23, 67– 81.
  • [23] Ma, S., Yang, L. and Carroll, R. (2012), ‘Simultaneous confidence band for sparse longitudinal regression’, Statistica Sinica 22, 95–122.
  • [24] Ma, S., Lian, H., Liang H. and Carroll, R. J. (2017). Supplemental Material: Additional Results for Simulation Studies. DOI:, 10.1214/17-EJS1291SUPP.
  • [25] Opsomer, J. and Ruppert, D. (1999), ‘A root-$n$ consistent backfitting estimator for semiparametric additive modeling’, Journal of Computational and Graphical Statistics 8, 715–732.
  • [26] Shen, X., Wolfe, D. A. and Zhou, S. (1998), ‘Local asymptotics for regression splines and confidence regions’, Annals of Statistics 26, 1760–1782.
  • [27] Song, Q. and Yang, L. (2010), ‘Oracally efficient spline smoothing of nonlinear additive autoregression model with simultaneous confidence band’, Journal of Multivariate Analysis 101, 2008–2025.
  • [28] Stone, C. (1986), ‘The dimensionality reduction principle for generalized additive models’, The Annals of Statistics 14, 590–606.
  • [29] Stone, C. J. (1985), ‘Additive regression and other nonparametric models’, The Annals of Statistics 13, 689–705.
  • [30] Stone, C. J. (1994), ‘The use of polynomial splines and their tensor products in multivariate function estimation’, The Annals of Statistics 22, 118–184.
  • [31] Wang, J.-L., Xue, L., Zhu, L. and Chong, Y. S. (2010), ‘Estimation for a partial-linear single-index model’, The Annals of Statistics 38, 246–274.
  • [32] Wang, L., Liu, X., Liang, H. and Carroll, R. (2011), ‘Estimation and variable selection for generalized additive partial linear models’, The Annals of Statistics 39, 1827–1851.
  • [33] Wang, L., Xue, L., Qu, A. and Liang, H. (2014), ‘Estimation and model selection in generalized additive partial linear models for high-dimensional correlated data’, The Annals of Statistics 42, 592–624.
  • [34] Wang, L. and Yang, L. (2007), ‘Spline-backfitted kernel smoothing of nonlinear additive autoregression model’, The Annals of Statistics 35, 2474–2503.
  • [35] Wood, S. N. (2006), Generalized Additive Models, Texts in Statistical Science Series, Chapman & Hall/CRC, Boca Raton, FL.
  • [36] Xia, Y. C. and Härdle, W. (2006), ‘Semi-parametric estimation of partially linear single-index models’, Journal of Multivariate Analysis 97, 1162–1184.
  • [37] Xia, Y. and Li, W. K. (1999), ‘On single-index coefficient regression models’, Journal of the American Statistical Association 94, 1275–1285.
  • [38] Xia, Y., Tong, H. and Li, W. K. (1999), ‘On extended partially linear single-index models’, Biometrika 86, 831–842.
  • [39] Xue, L., Qu, A. and Zhou, J. (2010), ‘Consistent model selection for marginal generalized additive model for correlated data’, Journal of the American Statistical Association 105(492), 1518–1530.
  • [40] Xue, L. and Yang, L. (2006), ‘Additive coefficient modeling via polynomial spline’, Statistica Sinica 16, 1423–1446.
  • [41] Zhou, S., Shen, X. and Wolfe, D. A. (1998), ‘Local asymptotics for regression splines and confidence regions’, Annals of Statistics 26, 1760–1782.
  • [42] Zhou, S. and Wolfe, D. A. (2000), ‘On derivative estimation in spline regression’, Statistica Sinica 10, 93–105.

Supplemental materials