Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions
Joel L. Horowitz and Enno Mammen
Source: Ann. Statist. Volume 35, Number 6
(2007), 2589-2619.
Abstract
This paper discusses a nonparametric regression model that naturally generalizes neural network models. The model is based on a finite number of one-dimensional transformations and can be estimated with a one-dimensional rate of convergence. The model contains the generalized additive model with unknown link function as a special case. For this case, it is shown that the additive components and link function can be estimated with the optimal rate by a smoothing spline that is the solution of a penalized least squares criterion.
First Page:
Show
Hide
Keywords: Generalized additive models; multivariate curve estimation; nonparametric regression; empirical process methods; penalized least squares; smoothing splines
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.aos/1201012973
Digital Object Identifier: doi:10.1214/009053607000000415
Mathematical Reviews number (MathSciNet): MR2382659
Zentralblatt MATH identifier: 1129.62034
References
Adams, R. A. (1975). Sobolev Spaces. Academic Press, New York.
Mathematical Reviews (MathSciNet): MR0450957
Zentralblatt MATH: 0314.46030
Agmon, S. (1965). Lectures on Elliptic Boundary Value Problems. Van Nostrand, Princeton, NJ.
Mathematical Reviews (MathSciNet): MR0178246
Zentralblatt MATH: 0142.37401
Birman, M. Š. and Solomjak, M. J. (1967). Piecewise polynomial approximation of functions of the classes $W_p^\alpha$. Mat. Sb. (N.S.) 73 295--317.
Mathematical Reviews (MathSciNet): MR0217487
Breiman, L. and Friedman, J. H. (1985). Estimating optimal transformations for multiple regression and correlation (with discussion). J. Amer. Statist. Assoc. 80 580--619.
Mathematical Reviews (MathSciNet): MR0803258
Digital Object Identifier: doi:10.2307/2288473
JSTOR: links.jstor.org
Zentralblatt MATH: 0594.62044
Buja, A., Hastie, T. and Tibshirani, R. (1989). Linear smoothers and additive models (with discussion). Ann. Statist. 17 453--555.
Mathematical Reviews (MathSciNet): MR0994249
Digital Object Identifier: doi:10.1214/aos/1176347115
Project Euclid: euclid.aos/1176347115
Zentralblatt MATH: 0689.62029
Christopeit, N. and Hoderlein, S. (2006). Local partitioned regression. Econometrica 74 787--817.
Mathematical Reviews (MathSciNet): MR2217617
Digital Object Identifier: doi:10.1111/j.1468-0262.2006.00683.x
Zentralblatt MATH: 1128.62047
Coppejans, M. (2004). On Kolmogorovs representation of functions of several variables by functions of one variable. J. Econometrics 123 1--31.
Mathematical Reviews (MathSciNet): MR2125436
Digital Object Identifier: doi:10.1016/j.jeconom.2003.10.026
Eubank, R. L. (1988). Spline Smoothing and Nonparametric Regression. Dekker, New York.
Mathematical Reviews (MathSciNet): MR0934016
Zentralblatt MATH: 0702.62036
Fan, J., Härdle, W. and Mammen, E. (1998). Direct estimation of low-dimensional components in additive models. Ann. Statist. 26 943--971.
Mathematical Reviews (MathSciNet): MR1635422
Digital Object Identifier: doi:10.1214/aos/1024691083
Project Euclid: euclid.aos/1024691083
Zentralblatt MATH: 1073.62527
Härdle, W., Huet, S., Mammen, E. and Sperlich, S. (2004). Bootstrap inference in semiparametric generalized additive models. Econometric Theory 20 265--300.
Mathematical Reviews (MathSciNet): MR2044272
Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman and Hall, London.
Mathematical Reviews (MathSciNet): MR1082147
Zentralblatt MATH: 0747.62061
Horowitz, J. (2001). Nonparametric estimation of a generalized additive model with an unknown link function. Econometrica 69 499--513.
Mathematical Reviews (MathSciNet): MR1819761
Digital Object Identifier: doi:10.1111/1468-0262.00200
JSTOR: links.jstor.org
Zentralblatt MATH: 0999.62032
Horowitz, J., Klemelä, J. and Mammen, E. (2006). Optimal estimation in additive regression models. Bernoulli 12 271--298.
Mathematical Reviews (MathSciNet): MR2218556
Digital Object Identifier: doi:10.3150/bj/1145993975
Project Euclid: euclid.bj/1145993975
Zentralblatt MATH: 1098.62043
Horowitz, J. and Mammen, E. (2004). Nonparametric estimation of an additive model with a link function. Ann. Statist. 32 2412--2443.
Mathematical Reviews (MathSciNet): MR2153990
Digital Object Identifier: doi:10.1214/009053604000000814
Project Euclid: euclid.aos/1107794874
Zentralblatt MATH: 1069.62035
Horowitz, J. and Mammen, E. (2006). Nonparametric estimation of a generalized additive model with an unknown link function. Preprint.
Ichimura, H. (1993). Semiparametric least squares (SLS) and weighted SLS estimation of single-index models. J. Econometrics 58 71--120.
Mathematical Reviews (MathSciNet): MR1230981
Digital Object Identifier: doi:10.1016/0304-4076(93)90114-K
Zentralblatt MATH: 0816.62079
Juditsky, A. B., Lepski, O. V. and Tsybakov, A. B. (2007). Nonparametric estimation of composite functions. Preprint.
Mathematical Reviews (MathSciNet): MR2509077
Zentralblatt MATH: 1160.62030
Digital Object Identifier: doi:10.1214/08-AOS611
Project Euclid: euclid.aos/1239369025
Kauermann, G. and Opsomer, J. D. (2003). Local likelihood estimation in generalized additive models. Scand. J. Statist. 30 317--337.
Mathematical Reviews (MathSciNet): MR1983128
Digital Object Identifier: doi:10.1111/1467-9469.00333
Zentralblatt MATH: 1053.62084
Linton, O. (2000). Efficient estimation of generalized additive nonparametric regression models. Econometric Theory 16 502--523.
Mathematical Reviews (MathSciNet): MR1790289
Digital Object Identifier: doi:10.1017/S0266466600164023
JSTOR: links.jstor.org
Zentralblatt MATH: 0963.62037
Linton, O. and Härdle, W. (1996). Estimation of additive regression models with known links. Biometrika 83 529--540.
Mathematical Reviews (MathSciNet): MR1423873
Zentralblatt MATH: 0866.62017
Digital Object Identifier: doi:10.1093/biomet/83.3.529
JSTOR: links.jstor.org
Linton, O. and Nielsen, J. P. (1995). A kernel method of estimating structured nonparametric regression based on marginal integration. Biometrika 82 93--100.
Mathematical Reviews (MathSciNet): MR1332841
Zentralblatt MATH: 0823.62036
Digital Object Identifier: doi:10.1093/biomet/82.1.93
JSTOR: links.jstor.org
Mammen, E., Linton, O. and Nielsen, J. P. (1999). The existence and asymptotic properties of a backfitting projection algorithm under weak conditions. Ann. Statist. 27 1443--1490.
Mathematical Reviews (MathSciNet): MR1742496
Project Euclid: euclid.aos/1017939138
Mammen, E. and Nielsen, J. P. (2003). Generalised structured models. Biometrika 90 551--566.
Mathematical Reviews (MathSciNet): MR2006834
Digital Object Identifier: doi:10.1093/biomet/90.3.551
Mammen, E. and Park, B. U. (2005). Bandwidth selection for smooth backfitting in additive models. Ann. Statist. 33 1260--1294.
Mathematical Reviews (MathSciNet): MR2195635
Digital Object Identifier: doi:10.1214/009053605000000101
Project Euclid: euclid.aos/1120224102
Zentralblatt MATH: 1072.62025
Mammen, E. and Park, B. U. (2006). A simple smooth backfitting method for additive models. Ann. Statist. 34 2252--2271.
Mathematical Reviews (MathSciNet): MR2291499
Digital Object Identifier: doi:10.1214/009053606000000696
Project Euclid: euclid.aos/1169571796
Zentralblatt MATH: 1106.62042
Mammen, E. and Thomas-Agnan, C. (1999). Smoothing splines and shape restrictions. Scand. J. Statist. 26 239--252.
Mathematical Reviews (MathSciNet): MR1707587
Digital Object Identifier: doi:10.1111/1467-9469.00147
Zentralblatt MATH: 0932.62051
Mammen, E. and van de Geer, S. (1997). Penalized quasi-likelihood estimation in partial linear models. Ann. Statist. 25 1014--1035.
Mathematical Reviews (MathSciNet): MR1447739
Digital Object Identifier: doi:10.1214/aos/1069362736
Project Euclid: euclid.aos/1069362736
Zentralblatt MATH: 0906.62033
McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd ed. Chapman and Hall, London.
Mathematical Reviews (MathSciNet): MR727836
Zentralblatt MATH: 0588.62104
Nelder, J. A. and Wedderburn, R. W. M. (1972). Generalized linear models. J. Roy. Statist. Soc. Ser. A 135 370--384.
Newey, W. K. (1997). Convergence rates and asymptotic normality for series estimators. J. Econometrics 79 147--168.
Mathematical Reviews (MathSciNet): MR1457700
Digital Object Identifier: doi:10.1016/S0304-4076(97)00011-0
Zentralblatt MATH: 0873.62049
Nielsen, J. P. and Sperlich, S. (2005). Smooth backfitting in practice. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 43--61.
Mathematical Reviews (MathSciNet): MR2136638
Digital Object Identifier: doi:10.1111/j.1467-9868.2005.00487.x
Zentralblatt MATH: 1060.62048
Oden, J. T. and Reddy, J. N. (1976). An Introduction to the Mathematical Theory of Finite Elements. Wiley, New York.
Mathematical Reviews (MathSciNet): MR0461950
Zentralblatt MATH: 0336.35001
Opsomer, J. D. (2000). Asymptotic properties of backfitting estimators. J. Multivariate Anal. 73 166--179.
Mathematical Reviews (MathSciNet): MR1763322
Digital Object Identifier: doi:10.1006/jmva.1999.1868
Zentralblatt MATH: 1065.62506
Opsomer, J. D. and Ruppert, D. (1997). Fitting a bivariate additive model by local polynomial regression. Ann. Statist. 25 186--211.
Mathematical Reviews (MathSciNet): MR1429922
Digital Object Identifier: doi:10.1214/aos/1034276626
Project Euclid: euclid.aos/1034276626
Zentralblatt MATH: 0869.62026
Stone, C. J. (1985). Additive regression and other nonparametric models. Ann. Statist. 13 689--705.
Mathematical Reviews (MathSciNet): MR0790566
Digital Object Identifier: doi:10.1214/aos/1176349548
Project Euclid: euclid.aos/1176349548
Zentralblatt MATH: 0605.62065
Stone, C. J. (1986). The dimensionality reduction principle for generalized additive models. Ann. Statist. 14 590--606.
Mathematical Reviews (MathSciNet): MR0840516
Digital Object Identifier: doi:10.1214/aos/1176349940
Project Euclid: euclid.aos/1176349940
Zentralblatt MATH: 0603.62050
Stone, C. J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation (with discussion). Ann. Statist. 22 118--184.
Mathematical Reviews (MathSciNet): MR1272079
Digital Object Identifier: doi:10.1214/aos/1176325361
Project Euclid: euclid.aos/1176325361
Zentralblatt MATH: 0827.62038
Tjøstheim, D. and Auestad, B. H. (1994). Nonparametric identification of nonlinear time series: Projections. J. Amer. Statist. Assoc. 89 1398--1409.
Mathematical Reviews (MathSciNet): MR1310230
Digital Object Identifier: doi:10.2307/2291002
JSTOR: links.jstor.org
Zentralblatt MATH: 0813.62036
van de Geer, S. (1990). Estimating a regression function. Ann. Statist. 18 907--924.
Mathematical Reviews (MathSciNet): MR1056343
Digital Object Identifier: doi:10.1214/aos/1176347632
Project Euclid: euclid.aos/1176347632
Zentralblatt MATH: 0709.62040
van de Geer, S. (2000). Empirical Processes in $M$-Estimation. Cambridge Univ. Press.
Wedderburn, R. W. M. (1974). Quasi-likelihood functions, generalized linear models and the Gauss--Newton method. Biometrika 61 439--447.
Yosida, K. (1974). Functional Analysis, 4th ed. Springer, New York.
Mathematical Reviews (MathSciNet): MR0350358
Zentralblatt MATH: 0286.46002
Yu, K., Park, B. U. and Mammen, E. (2007). Smooth backfitting in generalized additive models. Ann. Statist. To appear.
Mathematical Reviews (MathSciNet): MR2387970
Digital Object Identifier: doi:10.1214/009053607000000596
Project Euclid: euclid.aos/1201877300
Zentralblatt MATH: 1132.62028
