The Annals of Statistics

A reproducing kernel Hilbert space approach to functional linear regression

Ming Yuan and T. Tony Cai

Full-text: Open access


We study in this paper a smoothness regularization method for functional linear regression and provide a unified treatment for both the prediction and estimation problems. By developing a tool on simultaneous diagonalization of two positive definite kernels, we obtain shaper results on the minimax rates of convergence and show that smoothness regularized estimators achieve the optimal rates of convergence for both prediction and estimation under conditions weaker than those for the functional principal components based methods developed in the literature. Despite the generality of the method of regularization, we show that the procedure is easily implementable. Numerical results are obtained to illustrate the merits of the method and to demonstrate the theoretical developments.

Article information

Ann. Statist., Volume 38, Number 6 (2010), 3412-3444.

First available in Project Euclid: 30 November 2010

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J05: Linear regression
Secondary: 62G20: Asymptotic properties

Covariance eigenfunction eigenvalue functional linear regression minimax optimal convergence rate principal component analysis reproducing kernel Hilbert space Sacks–Ylvisaker conditions simultaneous diagonalization slope function Sobolev space


Yuan, Ming; Cai, T. Tony. A reproducing kernel Hilbert space approach to functional linear regression. Ann. Statist. 38 (2010), no. 6, 3412--3444. doi:10.1214/09-AOS772.

Export citation


  • Adams, R. A. (1975). Sobolev Spaces. Academic Press, New York.
  • Cai, T. and Hall, P. (2006). Prediction in functional linear regression. Ann. Statist. 34 2159–2179.
  • Cardot, H., Ferraty, F. and Sarda, P. (2003). Spline estimators for the functional linear model. Statist. Sinica 13 571–591.
  • Cardot, H. and Sarda, P. (2005). Estimation in generalized linear models for functional data via penalized likelihood. J. Multivariate Anal. 92 24–41.
  • Cox, D. D. and O’Sullivan, F. (1990). Asymptotic analysis of penalized likelihood and related estimators. Ann. Statist. 18 1676–1695.
  • Crambes, C., Kneip, A. and Sarda, P. (2009). Smoothing splines estimators for functional linear regression. Ann. Statist. 37 35–72.
  • Cucker, F. and Smale, S. (2001). On the mathematical foundations of learning. Bull. Amer. Math. Soc. 39 1–49.
  • Ferraty, F. and Vieu, P. (2006). Nonparametric Functional Data Analysis: Methods, Theory, Applications and Implementations. Springer, New York.
  • Hall, P. and Horowitz, J. L. (2007). Methodology and convergence rates for functional linear regression. Ann. Statist. 35 70–91.
  • James, G. (2002). Generalized linear models with functional predictors. J. Roy. Statist. Soc. Ser. B 64 411–432.
  • Johannes, J. (2009). Nonparametric estimation in functional linear models with second order stationary regressors. Unpublished manuscript. Available at
  • Li, Y. and Hsing, T. (2007). On the rates of convergence in functional linear regression. J. Multivariate Anal. 98 1782–1804.
  • Micchelli, C. and Wahba, G. (1981). Design problems for optimal surface interpolation. In Approximation Theory and Applications (Z. Ziegler, ed.) 329–347. Academic Press, New York.
  • Müller, H. G. and Stadtmüller, U. (2005). Generalized functional linear models. Ann. Statist. 33 774–805.
  • Ramsay, J. O. and Silverman, B. W. (2005). Functional Data Analysis, 2nd ed. Springer, New York.
  • Riesz, F. and Sz-Nagy, B. (1955). Functional Analysis. Ungar, New York.
  • Ritter, K., Wasilkowski, G. and Woźniakowski, H. (1995). Multivariate integeration and approximation for random fields satisfying Sacks–Ylvisaker conditions. Ann. Appl. Probab. 5 518–540.
  • Sacks, J. and Ylvisaker, D. (1966). Designs for regression problems with correlated errors. Ann. Math. Statist. 37 66–89.
  • Sacks, J. and Ylvisaker, D. (1968). Designs for regression problems with correlated errors; many parameters. Ann. Math. Statist. 39 49–69.
  • Sacks, J. and Ylvisaker, D. (1970). Designs for regression problems with correlated errors III. Ann. Math. Statist. 41 2057–2074.
  • Silverman, B. W. (1982). On the estimation of a probability density function by the maximum penalized likelihood method. Ann. Statist. 10 795–810.
  • Stein, M. (1999). Statistical Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York.
  • Wahba, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia.
  • Yao, F., Müller, H. G. and Wang, J. L. (2005). Functional linear regression analysis for longitudinal data. Ann. Statist. 33 2873–2903.