The Annals of Statistics

Adaptive functional linear regression

Fabienne Comte and Jan Johannes

Full-text: Open access


We consider the estimation of the slope function in functional linear regression, where scalar responses are modeled in dependence of random functions. Cardot and Johannes [J. Multivariate Anal. 101 (2010) 395–408] have shown that a thresholded projection estimator can attain up to a constant minimax-rates of convergence in a general framework which allows us to cover the prediction problem with respect to the mean squared prediction error as well as the estimation of the slope function and its derivatives. This estimation procedure, however, requires an optimal choice of a tuning parameter with regard to certain characteristics of the slope function and the covariance operator associated with the functional regressor. As this information is usually inaccessible in practice, we investigate a fully data-driven choice of the tuning parameter which combines model selection and Lepski’s method. It is inspired by the recent work of Goldenshluger and Lepski [Ann. Statist. 39 (2011) 1608–1632]. The tuning parameter is selected as minimizer of a stochastic penalized contrast function imitating Lepski’s method among a random collection of admissible values. This choice of the tuning parameter depends only on the data and we show that within the general framework the resulting data-driven thresholded projection estimator can attain minimax-rates up to a constant over a variety of classes of slope functions and covariance operators. The results are illustrated considering different configurations which cover in particular the prediction problem as well as the estimation of the slope and its derivatives. A simulation study shows the reasonable performance of the fully data-driven estimation procedure.

Article information

Ann. Statist., Volume 40, Number 6 (2012), 2765-2797.

First available in Project Euclid: 8 February 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J05: Linear regression
Secondary: 62G35: Robustness 62G20: Asymptotic properties

Adaptation model selestion Lepski’s method linear Galerkin approach prediction derivative estimation minimax theory


Comte, Fabienne; Johannes, Jan. Adaptive functional linear regression. Ann. Statist. 40 (2012), no. 6, 2765--2797. doi:10.1214/12-AOS1050.

Export citation


  • [1] Barron, A., Birgé, L. and Massart, P. (1999). Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 301–413.
  • [2] Bosq, D. (2000). Linear Processes in Function Spaces: Theory and Applications. Lecture Notes in Statistics 149. Springer, New York.
  • [3] Butucea, C. and Tsybakov, A. B. (2007). Sharp optimality in density deconvolution with dominating bias. I. Teor. Veroyatn. Primen. 52 111–128.
  • [4] Butucea, C. and Tsybakov, A. B. (2007). Sharp optimality in density deconvolution with dominating bias. II. Teor. Veroyatn. Primen. 52 336–349.
  • [5] Cai, T. and Zhou, H. (2008). Adaptive functional linear regression. Technical report. Available at
  • [6] Cardot, H., Ferraty, F. and Sarda, P. (1999). Functional linear model. Statist. Probab. Lett. 45 11–22.
  • [7] Cardot, H., Ferraty, F. and Sarda, P. (2003). Spline estimators for the functional linear model. Statist. Sinica 13 571–591.
  • [8] Cardot, H. and Johannes, J. (2010). Thresholding projection estimators in functional linear models. J. Multivariate Anal. 101 395–408.
  • [9] Cardot, H., Mas, A. and Sarda, P. (2007). CLT in functional linear regression models. Probab. Theory Related Fields 138 325–361.
  • [10] Comte, F. and Johannes, J. (2010). Adaptive estimation in circular functional linear models. Math. Methods Statist. 19 42–63.
  • [11] Comte, F., Rozenholc, Y. and Taupin, M.-L. (2006). Penalized contrast estimator for adaptive density deconvolution. Canad. J. Statist. 34 431–452.
  • [12] Crambes, C., Kneip, A. and Sarda, P. (2009). Smoothing splines estimators for functional linear regression. Ann. Statist. 37 35–72.
  • [13] Efromovich, S. and Koltchinskii, V. (2001). On inverse problems with unknown operators. IEEE Trans. Inform. Theory 47 2876–2894.
  • [14] Engl, H. W., Hanke, M. and Neubauer, A. (2000). Regularization of Inverse Problems. Mathematics and Its Applications 375. Kluwer Academic Press, Dordrecht.
  • [15] Ferraty, F. and Vieu, P. (2006). Nonparametric Functional Data Analysis: Theory and Practice. Springer, New York.
  • [16] Forni, M. and Reichlin, L. (1998). Let’s get real: A factor analytical approach to disaggregated business cycle dynamics. Rev. Econom. Stud. 65 453–473.
  • [17] Frank, I. and Friedman, J. (1993). A statistical view of some chemometrics regression tools. Technometrics 35 109–148.
  • [18] Goldenshluger, A. and Lepski, O. (2011). Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality. Ann. Statist. 39 1608–1632.
  • [19] Goldenshluger, A. and Tsybakov, A. (2001). Adaptive prediction and estimation in linear regression with infinitely many parameters. Ann. Statist. 29 1601–1619.
  • [20] Goldenshluger, A. and Tsybakov, A. (2003). Optimal prediction for linear regression with infinitely many parameters. J. Multivariate Anal. 84 40–60.
  • [21] Hall, P. and Horowitz, J. L. (2007). Methodology and convergence rates for functional linear regression. Ann. Statist. 35 70–91.
  • [22] Heinz, E. (1951). Beiträge zur Störungstheorie der Spektralzerlegung. Math. Ann. 123 415–438.
  • [23] Hoffmann, M. and Reiss, M. (2008). Nonlinear estimation for linear inverse problems with error in the operator. Ann. Statist. 36 310–336.
  • [24] Johannes, J. and Schenk, R. (2010). On rate optimal local estimation in functional linear model. Univ. Catholique de Louvain. Available at
  • [25] Kawata, T. (1972). Fourier Analysis in Probability Theory. Probability and Mathematical Statistics 15. Academic Press, New York.
  • [26] Klein, T. and Rio, E. (2005). Concentration around the mean for maxima of empirical processes. Ann. Probab. 33 1060–1077.
  • [27] Lepskiĭ, O. V. (1990). A problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl. 35 454–466.
  • [28] Marx, B. D. and Eilers, P. H. (1999). Generalized linear regression on sampled signals and curves: A p-spline approach. Technometrics 41 1–13.
  • [29] Massart, P. (2007). Concentration Inequalities and Model Selection. Lecture Notes in Math. 1896. Springer, Berlin. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6–23, 2003, with a foreword by Jean Picard.
  • [30] Mathé, P. (2006). The Lepskiĭ principle revisited. Inverse Problems 22 L11–L15.
  • [31] Müller, H.-G. and Stadtmüller, U. (2005). Generalized functional linear models. Ann. Statist. 33 774–805.
  • [32] Natterer, F. (1984). Error bounds for Tikhonov regularization in Hilbert scales. Appl. Anal. 18 29–37.
  • [33] Neubauer, A. (1988). When do Sobolev spaces form a Hilbert scale? Proc. Amer. Math. Soc. 103 557–562.
  • [34] Petrov, V. V. (1995). Limit Theorems of Probability Theory: Sequences of Independent Random Variables. Oxford Studies in Probability 4. Clarendon Press, Oxford.
  • [35] Preda, C. and Saporta, G. (2005). PLS regression on a stochastic process. Comput. Statist. Data Anal. 48 149–158.
  • [36] Ramsay, J. O. and Silverman, B. W. (2005). Functional Data Analysis, 2nd ed. Springer, New York.
  • [37] Talagrand, M. (1996). New concentration inequalities in product spaces. Invent. Math. 126 505–563.
  • [38] Verzelen, N. (2010). High-dimensional Gaussian model selection on a Gaussian design. Ann. Inst. Henri Poincaré Probab. Stat. 46 480–524.

Supplemental materials