The Annals of Statistics

A reproducing kernel Hilbert space approach to functional linear regression

Ming Yuan and T. Tony Cai

Full-text: Open access

Abstract

We study in this paper a smoothness regularization method for functional linear regression and provide a unified treatment for both the prediction and estimation problems. By developing a tool on simultaneous diagonalization of two positive definite kernels, we obtain shaper results on the minimax rates of convergence and show that smoothness regularized estimators achieve the optimal rates of convergence for both prediction and estimation under conditions weaker than those for the functional principal components based methods developed in the literature. Despite the generality of the method of regularization, we show that the procedure is easily implementable. Numerical results are obtained to illustrate the merits of the method and to demonstrate the theoretical developments.

Article information

Source
Ann. Statist., Volume 38, Number 6 (2010), 3412-3444.

Dates
First available in Project Euclid: 30 November 2010

Permanent link to this document
https://projecteuclid.org/euclid.aos/1291126962

Digital Object Identifier
doi:10.1214/09-AOS772

Mathematical Reviews number (MathSciNet)
MR2766857

Zentralblatt MATH identifier
1204.62074

Subjects
Primary: 62J05: Linear regression
Secondary: 62G20: Asymptotic properties

Keywords
Covariance eigenfunction eigenvalue functional linear regression minimax optimal convergence rate principal component analysis reproducing kernel Hilbert space Sacks–Ylvisaker conditions simultaneous diagonalization slope function Sobolev space

Citation

Yuan, Ming; Cai, T. Tony. A reproducing kernel Hilbert space approach to functional linear regression. Ann. Statist. 38 (2010), no. 6, 3412--3444. doi:10.1214/09-AOS772. https://projecteuclid.org/euclid.aos/1291126962


Export citation

References

  • Adams, R. A. (1975). Sobolev Spaces. Academic Press, New York.
  • Cai, T. and Hall, P. (2006). Prediction in functional linear regression. Ann. Statist. 34 2159–2179.
  • Cardot, H., Ferraty, F. and Sarda, P. (2003). Spline estimators for the functional linear model. Statist. Sinica 13 571–591.
  • Cardot, H. and Sarda, P. (2005). Estimation in generalized linear models for functional data via penalized likelihood. J. Multivariate Anal. 92 24–41.
  • Cox, D. D. and O’Sullivan, F. (1990). Asymptotic analysis of penalized likelihood and related estimators. Ann. Statist. 18 1676–1695.
  • Crambes, C., Kneip, A. and Sarda, P. (2009). Smoothing splines estimators for functional linear regression. Ann. Statist. 37 35–72.
  • Cucker, F. and Smale, S. (2001). On the mathematical foundations of learning. Bull. Amer. Math. Soc. 39 1–49.
  • Ferraty, F. and Vieu, P. (2006). Nonparametric Functional Data Analysis: Methods, Theory, Applications and Implementations. Springer, New York.
  • Hall, P. and Horowitz, J. L. (2007). Methodology and convergence rates for functional linear regression. Ann. Statist. 35 70–91.
  • James, G. (2002). Generalized linear models with functional predictors. J. Roy. Statist. Soc. Ser. B 64 411–432.
  • Johannes, J. (2009). Nonparametric estimation in functional linear models with second order stationary regressors. Unpublished manuscript. Available at http://arxiv.org/abs/0901.4266v1.
  • Li, Y. and Hsing, T. (2007). On the rates of convergence in functional linear regression. J. Multivariate Anal. 98 1782–1804.
  • Micchelli, C. and Wahba, G. (1981). Design problems for optimal surface interpolation. In Approximation Theory and Applications (Z. Ziegler, ed.) 329–347. Academic Press, New York.
  • Müller, H. G. and Stadtmüller, U. (2005). Generalized functional linear models. Ann. Statist. 33 774–805.
  • Ramsay, J. O. and Silverman, B. W. (2005). Functional Data Analysis, 2nd ed. Springer, New York.
  • Riesz, F. and Sz-Nagy, B. (1955). Functional Analysis. Ungar, New York.
  • Ritter, K., Wasilkowski, G. and Woźniakowski, H. (1995). Multivariate integeration and approximation for random fields satisfying Sacks–Ylvisaker conditions. Ann. Appl. Probab. 5 518–540.
  • Sacks, J. and Ylvisaker, D. (1966). Designs for regression problems with correlated errors. Ann. Math. Statist. 37 66–89.
  • Sacks, J. and Ylvisaker, D. (1968). Designs for regression problems with correlated errors; many parameters. Ann. Math. Statist. 39 49–69.
  • Sacks, J. and Ylvisaker, D. (1970). Designs for regression problems with correlated errors III. Ann. Math. Statist. 41 2057–2074.
  • Silverman, B. W. (1982). On the estimation of a probability density function by the maximum penalized likelihood method. Ann. Statist. 10 795–810.
  • Stein, M. (1999). Statistical Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York.
  • Wahba, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia.
  • Yao, F., Müller, H. G. and Wang, J. L. (2005). Functional linear regression analysis for longitudinal data. Ann. Statist. 33 2873–2903.