Electronic Journal of Statistics

Optimal prediction for additive function-on-function regression

Matthew Reimherr, Bharath Sriperumbudur, and Bahaeddine Taoufik

Full-text: Open access


As with classic statistics, functional regression models are invaluable in the analysis of functional data. While there are now extensive tools with accompanying theory available for linear models, there is still a great deal of work to be done concerning nonlinear models for functional data. In this work we consider the Additive Function-on-Function Regression model, a type of nonlinear model that uses an additive relationship between the functional outcome and functional covariate. We present an estimation methodology built upon Reproducing Kernel Hilbert Spaces, and establish optimal rates of convergence for our estimates in terms of prediction error. We also discuss computational challenges that arise with such complex models, developing a representer theorem for our estimate as well as a more practical and computationally efficient approximation. Simulations and an application to Cumulative Intraday Returns around the 2008 financial crisis are also provided.

Article information

Electron. J. Statist., Volume 12, Number 2 (2018), 4571-4601.

Received: September 2017
First available in Project Euclid: 21 December 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression 62G20: Asymptotic properties

Nonlinear regression functional data analysis reproducing kernel Hilbert space minimax convergence

Creative Commons Attribution 4.0 International License.


Reimherr, Matthew; Sriperumbudur, Bharath; Taoufik, Bahaeddine. Optimal prediction for additive function-on-function regression. Electron. J. Statist. 12 (2018), no. 2, 4571--4601. doi:10.1214/18-EJS1505. https://projecteuclid.org/euclid.ejs/1545382950

Export citation


  • [1] A. Berlinet and C. Thomas-Agnan., Reproducing Kernel Hilbert spaces in Probability and Statistics. Springer Science & Business Media, 2011.
  • [2] T. T. Cai and M. Yuan. Optimal estimation of the mean function based on discretely sampled functional data: Phase transition., The Annals of Statistics, 39(5) :2330–2355, 2011.
  • [3] T. T. Cai and M. Yuan. Minimax and adaptive prediction for functional linear regression., Journal of the American Statistical Association, 107(499) :1201–1216, 2012.
  • [4] P. Du and X. Wang. Penalized likelihood functional regression., Statistica Sinica, pages 1017–1041, 2014.
  • [5] Y. Fan, G. M. James, and P. Radchenko. Functional additive regression., The Annals of Statistics, 43(5) :2296–2325, 2015.
  • [6] R. Gabrys, L. Horváth, and P. Kokoszka. Tests for error correlation in the functional linear model., Journal of the American Statistical Association, 105(491) :1113–1125, 2010.
  • [7] T. Hastie and R. Tibshirani., Generalized Additive Models. Wiley Online Library, 1990.
  • [8] L. Horváth and P. Kokoszka., Inference for Functional Data with Applications, volume 200. Springer, 2012.
  • [9] L. Horváth, P. Kokoszka, and G. Rice. Testing stationarity of functional time series., Journal of Econometrics, 179(1):66–82, 2014.
  • [10] G. M. James and B. W. Silverman. Functional adaptive model estimation., Journal of the American Statistical Association, 100(470):565–576, 2005.
  • [11] H. Kadri, E. Duflos, P. Preux, S. Canu, and M. Davy. Nonlinear functional regression: A functional RKHS approach. In Y. W. Teh and M. Titterington, editors, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, volume 9 of Proceedings of Machine Learning Research, pages 374–380, 2010.
  • [12] R. A. Kennedy and P. Sadeghi., Hilbert Space Methods in Signal Processing. Cambridge University Press, 2013.
  • [13] J. S. Kim, A.-M. Staicu, A. Maity, R. J. Carroll, and D. Ruppert. Additive function-on-function regression., Journal of Computational and Graphical Statistics, 27:234–244, 2018.
  • [14] G. S. Kimeldorf and G. Wahba. Some results on Tchebycheffian spline functions., Journal of Mathematical Analysis and Applications, 33:82–95, 1971.
  • [15] P. Kokoszka and M. Reimherr. Predictability of shapes of intraday price curves., The Econometrics Journal, 16(3):285–308, 2013.
  • [16] P. Kokoszka and M. Reimherr., Introduction to Functional Data Analysis. CRC Press, 2017.
  • [17] Y. Li, T. Hsing, et al. Uniform convergence rates for nonparametric regression and principal component analysis in functional/longitudinal data., The Annals of Statistics, 38(6) :3321–3351, 2010.
  • [18] H. Lian. Nonlinear functional models for functional responses in reproducing kernel Hilbert spaces., Canadian Journal of Statistics, 35(4):597–606, 2007.
  • [19] H. Ma and Z. Zhu. Continuously dynamic additive models for functional data., Journal of Multivariate Analysis, 150:1–13, 2016.
  • [20] M. W. McLean, G. Hooker, A.-M. Staicu, F. Scheipl, and D. Ruppert. Functional generalized additive models., Journal of Computational and Graphical Statistics, 23(1):249–269, 2014.
  • [21] J. S. Morris. Functional regression., Annual Review of Statistics and Its Application, 2:321–359, 2015.
  • [22] H.-G. Müller, Y. Wu, and F. Yao. Continuously additive models for nonlinear functional regression., Biometrika, pages 607–622, 2013.
  • [23] J. G. Nicholas., Market Neutral Investing. Bloomberg Press Princeton, NJ, 2000.
  • [24] A. Parodi and M. Reimherr. FLAME: Simultaneous variable selection and smoothing for function-on-scalar regression. Technical report, Pennsylvania State University, 2017.
  • [25] J. Petrovich, M. Reimherr, and C. Daymont. Functional regression models with highly irregular designs. Technical report, Pennsylvania State University, 2018., (https://arxiv.org/abs/1805.08518).
  • [26] C. Preda. Regression models for functional data by reproducing kernel Hilbert spaces methods., Journal of Statistical Planning and Inference, 137(3):829–840, 2007.
  • [27] J. O. Ramsay and B. Silverman., Functional Data Analysis. Wiley Online Library, 2006.
  • [28] J. O. Ramsay, G. Hooker, and S. Graves., Functional Data Analysis with R and MATLAB. Springer Science & Business Media, 2009.
  • [29] F. Scheipl, A.-M. Staicu, and S. Greven. Functional additive mixed models., Journal of Computational and Graphical Statistics, 24(2):477–501, 2015.
  • [30] X. Sun, P. Du, X. Wang, and P. Ma. Optimal penalized function-on-function regression under a reproducing kernel Hilbert space framework., Journal of the American Statistical Association, page Accepted, 2017.
  • [31] K. G. van den Boogaart. tensorA: Advanced tensors arithmetic with named indices., R package version 0.31, 2007. http://CRAN.R-project.org/package=tensorA.
  • [32] X. Wang and D. Ruppert. Optimal prediction in an additive functional model., Statistica Sinica, 25:567–589, 2015.
  • [33] L. Xiao, C. Li, W. Checkley, and C. Crainiceanu. Fast covariance estimation for sparse functional data., Statistics and Computing, pages 1–12, 2017.
  • [34] F. Yao, H.-G. Müller, and J.-L. Wang. Functional data analysis for sparse longitudinal data., Journal of the American Statistical Association, 100(470):577–590, 2005.
  • [35] X. Zhang, J.-L. Wang, et al. From sparse to dense functional data and beyond., The Annals of Statistics, 44(5) :2281–2321, 2016.
  • [36] H. Zhu, F. Yao, and H. H. Zhang. Structured functional additive regression in reproducing kernel Hilbert spaces., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76(3):581–603, 2014.