Electronic Journal of Statistics

On rate optimal local estimation in functional linear regression

Jan Johannes and Rudolf Schenk

Full-text: Open access


We consider the estimation of the value of a linear functional of the slope parameter in functional linear regression, where scalar responses are modeled in dependence of random functions. The theory in this paper covers in particular point-wise estimation as well as the estimation of weighted averages of the slope parameter. We propose a plug-in estimator which is based on a dimension reduction technique and additional thresholding. It is shown that this estimator is consistent under mild assumptions. We derive a lower bound for the maximal mean squared error of any estimator over a certain ellipsoid of slope parameters and a certain class of covariance operators associated with the regressor. It is shown that the proposed estimator attains this lower bound up to a constant and hence it is minimax optimal. Our results are appropriate to discuss a wide range of possible regressors, slope parameters and functionals. They are illustrated by considering the point-wise estimation of the slope parameter or its derivatives and its average value over a given interval.

Article information

Electron. J. Statist., Volume 7 (2013), 191-216.

First available in Project Euclid: 24 January 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62J05: Linear regression
Secondary: 62G05: Estimation 62J20: Diagnostics

Linear functional linear Galerkin projection minimax-theory point-wise estimation Sobolev space thresholding


Johannes, Jan; Schenk, Rudolf. On rate optimal local estimation in functional linear regression. Electron. J. Statist. 7 (2013), 191--216. doi:10.1214/13-EJS767. https://projecteuclid.org/euclid.ejs/1359041589

Export citation


  • [1] Bosq, D., Linear Processes in Function Spaces., volume 149 of Lecture Notes in Statistics. Springer-Verlag, 2000.
  • [2] Cai, T. and Hall, P. Prediction in functional linear regression., Annals of Statistics, 34(5) :2159–2179, 2006.
  • [3] Cardot, H., Ferraty, F., and Sarda, P. Spline estimators for the functional linear model., Statistica Sinica, 13:571–591, 2003.
  • [4] Cardot, H. and Johannes, J. Thresholding projection estimators in functional linear models., Journal of Multivariate Analysis, 101(2):395–408, 2010.
  • [5] Cardot, H., Mas, A., and Sarda, P. CLT in functional linear regression models., Probability Theory and Related Fields, 138:325–361, 2007.
  • [6] Crambes, C., Kneip, A., and Sarda, P. Smoothing splines estimators for functional linear regression., Annals of Statistics, 37(1):35–72, 2009.
  • [7] Donoho, D. Statistical estimation and optimal recovery., Annals of Statistics, 22:238–270, 1994.
  • [8] Donoho, D. and Low, M. Renormalization exponents and optimal pointwise rates of convergence., Annals of Statistics, 20:944–970, 1992.
  • [9] Efromovich, S. and Koltchinskii, V. On inverse problems with unknown operators., IEEE Transactions on Information Theory, 47(7) :2876–2894, 2001.
  • [10] Eilers, P. H. and Marx, B. D. Flexible smoothing with B-splines and penalties., Statistical Science, 11:89–102, 1996.
  • [11] Goldenshluger, A. and Pereverzev, S. V. Adaptive estimation of linear functionals in Hilbert scales from indirect white noise observations., Probability Theory and Related Fields, 118:169–186, 2000.
  • [12] Hall, P. and Horowitz, J. L. Methodology and convergence rates for functional linear regression., Annals of Statistics, 35(1):70–91, 2007.
  • [13] Heinz, E. Beiträge zur Störungstheorie der Spektralzerlegung., Mathematische Annalen, 123:415–438, 1951.
  • [14] Hoffmann, M. and Reiß, M. Nonlinear estimation for linear inverse problems with error in the operator., Annals of Statistics, 36(1):310–336, 2008.
  • [15] Ibragimov, I. and Has’minskii, R. On nonparametric estimation of the value of a linear functional in Gaussian white noise., Theory of Probability and its Applications, 29:18–32, 1984.
  • [16] Johannes, J. and Kroll, M. On rate optimal estimation of linear functionals for linear inverse problems with error in the operator. Technical report, University Heidelberg, 2010.
  • [17] Kawata, T., Fourier analysis in probability theory. Academic Press, New York, 1972.
  • [18] Le Cam, L. Convergence of Estimates Under Dimensionality Restrictions., Annals of Statistics, 1(1):38–53, 1973.
  • [19] Li, K. Minimaxity of the method of regularization of stochastic processes., Annals of Statistics, 10:937–942, 1982.
  • [20] Mair, B. A. Tikhonov regularization for finitely and infinitely smoothing operators., SIAM Journal on Mathematical Analysis, 25:135–147, 1994.
  • [21] Meister, A. Asymptotic equivalence of functional linear regression and a white noise inverse problem., Annals of Statistics, 39(3) :1471–1495, 2011.
  • [22] Müller, H.-G. and Stadtmüller, U. Generalized functional linear models., Annals of Statistics, 33:774–805, 2005.
  • [23] Natterer, F. Error bounds for Tikhonov regularization in Hilbert scales., Applicable Analysis, 18:29–37, 1984.
  • [24] Neubauer, A. An a posteriori parameter choice for Tikhonov regularization in Hilbert scales leading to optimal convergence rates., SIAM Journal on Numerical Analysis, 25(6) :1313–1326, 1988a.
  • [25] Neubauer, A. When do Sobolev spaces form a Hilbert scale?, Proceedings of the American Mathematical Society, 103(2):557–562, 1988b.
  • [26] Petrov, V. V., Limit theorems of probability theory. Sequences of independent random variables. Oxford Studies in Probability. Clarendon Press, Oxford, 4. edition, 1995.
  • [27] Ramsay, J. O. and Dalzell, C. J. Some tools for functional data analysis., Journal of the Royal Statistical Society, Series B, 53:539–572, 1991.
  • [28] Speckman, P. Minimax estimation of linear functionals in a Hilbert space. Unpublished manuscript, 1979.