The Annals of Statistics

Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions

Joel L. Horowitz and Enno Mammen

Full-text: Open access


This paper discusses a nonparametric regression model that naturally generalizes neural network models. The model is based on a finite number of one-dimensional transformations and can be estimated with a one-dimensional rate of convergence. The model contains the generalized additive model with unknown link function as a special case. For this case, it is shown that the additive components and link function can be estimated with the optimal rate by a smoothing spline that is the solution of a penalized least squares criterion.

Article information

Ann. Statist. Volume 35, Number 6 (2007), 2589-2619.

First available: 22 January 2008

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression
Secondary: 62G20: Asymptotic properties

Generalized additive models multivariate curve estimation nonparametric regression empirical process methods penalized least squares smoothing splines


Horowitz, Joel L.; Mammen, Enno. Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions. The Annals of Statistics 35 (2007), no. 6, 2589--2619. doi:10.1214/009053607000000415.

Export citation


  • Adams, R. A. (1975). Sobolev Spaces. Academic Press, New York.
  • Agmon, S. (1965). Lectures on Elliptic Boundary Value Problems. Van Nostrand, Princeton, NJ.
  • Birman, M. Š. and Solomjak, M. J. (1967). Piecewise polynomial approximation of functions of the classes $W_p^\alpha$. Mat. Sb. (N.S.) 73 295–317.
  • Breiman, L. and Friedman, J. H. (1985). Estimating optimal transformations for multiple regression and correlation (with discussion). J. Amer. Statist. Assoc. 80 580–619.
  • Buja, A., Hastie, T. and Tibshirani, R. (1989). Linear smoothers and additive models (with discussion). Ann. Statist. 17 453–555.
  • Christopeit, N. and Hoderlein, S. (2006). Local partitioned regression. Econometrica 74 787–817.
  • Coppejans, M. (2004). On Kolmogorovs representation of functions of several variables by functions of one variable. J. Econometrics 123 1–31.
  • Eubank, R. L. (1988). Spline Smoothing and Nonparametric Regression. Dekker, New York.
  • Fan, J., Härdle, W. and Mammen, E. (1998). Direct estimation of low-dimensional components in additive models. Ann. Statist. 26 943–971.
  • Härdle, W., Huet, S., Mammen, E. and Sperlich, S. (2004). Bootstrap inference in semiparametric generalized additive models. Econometric Theory 20 265–300.
  • Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman and Hall, London.
  • Horowitz, J. (2001). Nonparametric estimation of a generalized additive model with an unknown link function. Econometrica 69 499–513.
  • Horowitz, J., Klemelä, J. and Mammen, E. (2006). Optimal estimation in additive regression models. Bernoulli 12 271–298.
  • Horowitz, J. and Mammen, E. (2004). Nonparametric estimation of an additive model with a link function. Ann. Statist. 32 2412–2443.
  • Horowitz, J. and Mammen, E. (2006). Nonparametric estimation of a generalized additive model with an unknown link function. Preprint.
  • Ichimura, H. (1993). Semiparametric least squares (SLS) and weighted SLS estimation of single-index models. J. Econometrics 58 71–120.
  • Juditsky, A. B., Lepski, O. V. and Tsybakov, A. B. (2007). Nonparametric estimation of composite functions. Preprint.
  • Kauermann, G. and Opsomer, J. D. (2003). Local likelihood estimation in generalized additive models. Scand. J. Statist. 30 317–337.
  • Linton, O. (2000). Efficient estimation of generalized additive nonparametric regression models. Econometric Theory 16 502–523.
  • Linton, O. and Härdle, W. (1996). Estimation of additive regression models with known links. Biometrika 83 529–540.
  • Linton, O. and Nielsen, J. P. (1995). A kernel method of estimating structured nonparametric regression based on marginal integration. Biometrika 82 93–100.
  • Mammen, E., Linton, O. and Nielsen, J. P. (1999). The existence and asymptotic properties of a backfitting projection algorithm under weak conditions. Ann. Statist. 27 1443–1490.
  • Mammen, E. and Nielsen, J. P. (2003). Generalised structured models. Biometrika 90 551–566.
  • Mammen, E. and Park, B. U. (2005). Bandwidth selection for smooth backfitting in additive models. Ann. Statist. 33 1260–1294.
  • Mammen, E. and Park, B. U. (2006). A simple smooth backfitting method for additive models. Ann. Statist. 34 2252–2271.
  • Mammen, E. and Thomas-Agnan, C. (1999). Smoothing splines and shape restrictions. Scand. J. Statist. 26 239–252.
  • Mammen, E. and van de Geer, S. (1997). Penalized quasi-likelihood estimation in partial linear models. Ann. Statist. 25 1014–1035.
  • McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd ed. Chapman and Hall, London.
  • Nelder, J. A. and Wedderburn, R. W. M. (1972). Generalized linear models. J. Roy. Statist. Soc. Ser. A 135 370–384.
  • Newey, W. K. (1997). Convergence rates and asymptotic normality for series estimators. J. Econometrics 79 147–168.
  • Nielsen, J. P. and Sperlich, S. (2005). Smooth backfitting in practice. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 43–61.
  • Oden, J. T. and Reddy, J. N. (1976). An Introduction to the Mathematical Theory of Finite Elements. Wiley, New York.
  • Opsomer, J. D. (2000). Asymptotic properties of backfitting estimators. J. Multivariate Anal. 73 166–179.
  • Opsomer, J. D. and Ruppert, D. (1997). Fitting a bivariate additive model by local polynomial regression. Ann. Statist. 25 186–211.
  • Stone, C. J. (1985). Additive regression and other nonparametric models. Ann. Statist. 13 689–705.
  • Stone, C. J. (1986). The dimensionality reduction principle for generalized additive models. Ann. Statist. 14 590–606.
  • Stone, C. J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation (with discussion). Ann. Statist. 22 118–184.
  • Tjøstheim, D. and Auestad, B. H. (1994). Nonparametric identification of nonlinear time series: Projections. J. Amer. Statist. Assoc. 89 1398–1409.
  • van de Geer, S. (1990). Estimating a regression function. Ann. Statist. 18 907–924.
  • van de Geer, S. (2000). Empirical Processes in $M$-Estimation. Cambridge Univ. Press.
  • Wedderburn, R. W. M. (1974). Quasi-likelihood functions, generalized linear models and the Gauss–Newton method. Biometrika 61 439–447.
  • Yosida, K. (1974). Functional Analysis, 4th ed. Springer, New York.
  • Yu, K., Park, B. U. and Mammen, E. (2007). Smooth backfitting in generalized additive models. Ann. Statist. To appear.