The Annals of Statistics

Nonparametric regression with nonparametrically generated covariates

Enno Mammen, Christoph Rothe, and Melanie Schienle

Full-text: Open access


We analyze the statistical properties of nonparametric regression estimators using covariates which are not directly observable, but have be estimated from data in a preliminary step. These so-called generated covariates appear in numerous applications, including two-stage nonparametric regression, estimation of simultaneous equation models or censored regression models. Yet so far there seems to be no general theory for their impact on the final estimator’s statistical properties. Our paper provides such results. We derive a stochastic expansion that characterizes the influence of the generation step on the final estimator, and use it to derive rates of consistency and asymptotic distributions accounting for the presence of generated covariates.

Article information

Ann. Statist. Volume 40, Number 2 (2012), 1132-1170.

First available in Project Euclid: 18 July 2012

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression 62G20: Asymptotic properties

Nonparametric regression two-stage estimators simultaneous equation models empirical process


Mammen, Enno; Rothe, Christoph; Schienle, Melanie. Nonparametric regression with nonparametrically generated covariates. Ann. Statist. 40 (2012), no. 2, 1132--1170. doi:10.1214/12-AOS995.

Export citation


  • Ahn, H. (1995). Nonparametric two-stage estimation of conditional choice probabilities in a binary choice model under uncertainty. J. Econometrics 67 337–378.
  • Andrews, D. W. K. (1994). Asymptotics for semiparametric econometric models via stochastic equicontinuity. Econometrica 62 43–72.
  • Andrews, D. W. K. (1995). Nonparametric kernel estimation for semiparametric models. Econometric Theory 11 560–596.
  • Blundell, R. W. and Powell, J. L. (2004). Endogeneity in semiparametric binary response models. Rev. Econom. Stud. 71 655–679.
  • Chen, X., Linton, O. and Van Keilegom, I. (2003). Estimation of semiparametric models when the criterion function is not smooth. Econometrica 71 1591–1608.
  • Conrad, C. and Mammen, E. (2009). Nonparametric regression on a generated covariate with an application to semiparametric GARCH-in-Mean models. Unpublished manuscript.
  • Das, M., Newey, W. K. and Vella, F. (2003). Nonparametric estimation of sample selection models. Rev. Econom. Stud. 70 33–58.
  • d’Haultfoeuille, X. and Maurel, A. (2009). Inference on a generalized Roy model, with an application to schooling decisions in France. Unpublished manuscript.
  • Einmahl, U. and Mason, D. M. (2000). An empirical process approach to the uniform consistency of kernel-type function estimators. J. Theoret. Probab. 13 1–37.
  • Escanciano, J. C., Jacho-Chávez, D. and Lewbel, A. (2011). Uniform convergence for semiparametric two step estimators and tests. Unpublished manuscript.
  • Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and Its Applications. CRC Press, New York.
  • Hahn, J. and Ridder, G. (2011). The asymptotic variance of semiparametric estimators with generated regressors. Unpublished manuscript.
  • Härdle, W., Janssen, P. and Serfling, R. (1988). Strong uniform consistency rates for estimators of conditional functionals. Ann. Statist. 16 1428–1449.
  • Heckman, J. J., Ichimura, H. and Todd, P. (1998). Matching as an econometric evaluation estimator. Rev. Econom. Stud. 65 261–294.
  • Heckman, J. J. and Vytlacil, E. (2005). Structural equations, treatment effects, and econometric policy evaluation. Econometrica 73 669–738.
  • Imbens, G. W. and Newey, W. K. (2009). Identification and estimation of triangular simultaneous equations models without additivity. Econometrica 77 1481–1512.
  • Kanaya, S. and Kristensen, D. (2009). Estimation of stochastic volatility models by nonparametric filtering. Unpublished manuscript.
  • Lewbel, A. and Linton, O. (2002). Nonparametric censored and truncated regression. Econometrica 70 765–779.
  • Li, Q. and Wooldridge, J. M. (2002). Semiparametric estimation of partially linear models for dependent data with generated regressors. Econometric Theory 18 625–645.
  • Linton, O. and Nielsen, J. P. (1995). A kernel method of estimating structured nonparametric regression based on marginal integration. Biometrika 82 93–100.
  • Mammen, E., Linton, O. and Nielsen, J. (1999). The existence and asymptotic properties of a backfitting projection algorithm under weak conditions. Ann. Statist. 27 1443–1490.
  • Mammen, E., Rothe, C. and Schienle, M. (2011). Semiparametric estimation with generated covariates. Unpublished manuscript.
  • Masry, E. (1996). Multivariate local polynomial regression for time series: Uniform strong consistency and rates. J. Time Ser. Anal. 17 571–599.
  • Newey, W. K. (1994a). Kernel estimation of partial means and a general variance estimator. Econometric Theory 10 233–253.
  • Newey, W. K. (1994b). The asymptotic variance of semiparametric estimators. Econometrica 62 1349–1382.
  • Newey, W. K. (1997). Convergence rates and asymptotic normality for series estimators. J. Econometrics 79 147–168.
  • Newey, W. K., Powell, J. L. and Vella, F. (1999). Nonparametric estimation of triangular simultaneous equations models. Econometrica 67 565–603.
  • Pagan, A. (1984). Econometric issues in the analysis of regressions with generated regressors. Internat. Econom. Rev. 25 221–247.
  • Song, K. (2008). Uniform convergence of series estimators over function spaces. Econometric Theory 24 1463–1499.
  • Sperlich, S. (2009). A note on non-parametric estimation with predicted variables. Econom. J. 12 382–395.
  • Stone, C. J. (1985). Additive regression and other nonparametric models. Ann. Statist. 13 689–705.
  • van de Geer, S. (2000). Empirical Processes in M-Estimation. Cambridge Univ. Press, Cambridge.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.