Electronic Journal of Statistics

Asymptotic theory of penalized splines

Luo Xiao

Full-text: Open access


The paper gives a unified study of the large sample asymptotic theory of penalized splines including the O-splines using B-splines and an integrated squared derivative penalty [22], the P-splines which use B-splines and a discrete difference penalty [13], and the T-splines which use truncated polynomials and a ridge penalty [24]. Extending existing results for O-splines [7], it is shown that, depending on the number of knots and appropriate smoothing parameters, the $L_{2}$ risk bounds of penalized spline estimators are rate-wise similar to either those of regression splines or to those of smoothing splines and could each attain the optimal minimax rate of convergence [32]. In addition, convergence rate of the $L_{\infty }$ risk bound, and local asymptotic bias and variance are derived for all three types of penalized splines.

Article information

Electron. J. Statist., Volume 13, Number 1 (2019), 747-794.

Received: September 2018
First available in Project Euclid: 21 March 2019

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression 62G20: Asymptotic properties

Nonparametric regression penalized splines $L_{\infty }$ convergence $L_{2}$ convergence local asymptotics rate optimality

Creative Commons Attribution 4.0 International License.


Xiao, Luo. Asymptotic theory of penalized splines. Electron. J. Statist. 13 (2019), no. 1, 747--794. doi:10.1214/19-EJS1541. https://projecteuclid.org/euclid.ejs/1553133771

Export citation


  • [1] Agarwal, G. G. and Studden, W. J. [1980], ‘Asymptotic integrated mean square error using least squares and bias minimizing splines’, Ann. Statist. 8(6), 1307–1325.
  • [2] Anderson, T. [1971], The Statistical Analysis of Time Series, John Wiley & Sons, Inc, New York.
  • [3] Barrow, D. L. and Smith, P. W. [1978], ‘Asymptotic properties of best $L_2$[0, 1] approximation by splines with variable knots’, Quarterly of Applied Mathematics 36(3), 293–304.
  • [4] Barrow, D. and Smith, P. [1979], ‘Efficient $L_2$ approximation by splines.’, Numerische Mathematik 33, 101–114.
  • [5] Chen, H., Wang, Y., Paik, M. C. and Choi, H. A. [2013], ‘A marginal approach to reduced-rank penalized spline smoothing with application to multilevel functional data’, Journal of the American Statistical Association 108(504), 1216–1229.
  • [6] Chen, X. and Christensen, T. M. [2015], ‘Optimal uniform convergence rates and asymptotic normality for series estimators under weak dependence and weak conditions’, Journal of Econometrics 188(2), 447 – 465. Heterogeneity in Panel Data and in Nonparametric Analysis in honor of Professor Cheng Hsiao.
  • [7] Claeskens, G., Krivobokova, T. and Opsomer, J. D. [2009], ‘Asymptotic properties of penalized spline estimators’, Biometrika 96(3), 529–544.
  • [8] de Boor, C. [1978], A Practical Guide to Splines, Springer, Berlin.
  • [9] Demko, S. [1977], ‘Inverses of band matrices and local convergence of spline projections’, SIAM Journal on Numerical Analysis 14(4), 616–619.
  • [10] Demko, S., Moss, W. F. and Smith, P. W. [1984], ‘Decay rates for inverses of band matrices’, Mathematics of Computation 43(168), 491–499.
  • [11] Durret, R. [2005], Probability: Theory and Examples, Third Edition, Thomson.
  • [12] Eggermont, P. P. B. and LaRiccia, V. N. [2006], Uniform error bounds for smoothing splines, Vol. Number 51 of Lecture Notes–Monograph Series, Institute of Mathematical Statistics, Beachwood, Ohio, USA, pp. 220–237.
  • [13] Eilers, P. and Marx, B. [1996], ‘Flexible smoothing with B-splines and penalties (with Discussion)’, Statist. Sci. 11, 89–121.
  • [14] Eilers, P., Marx, B. and Durban, M. [2015], ‘Twenty years of p-splines’, SORT 39(2), 1149–186.
  • [15] Goldsmith, J., Bobb, J., Crainiceanu, C. M., Caffo, B. and Reich, D. [2011], ‘Penalized functional regression’, Journal of Computational and Graphical Statistics 20(4), 830–851. PMID: 22368438.
  • [16] Gradshteyn, I. and Ryzhik, I. [2007], Table of Integrals, Series, and Products, Academic Press, New York.
  • [17] Hall, P. and Opsomer, J. D. [2005], ‘Theory for penalised spline regression’, Biometrika 92(1), 105–118.
  • [18] Kauermann, G., Krivobokova, T. and Fahrmeir, L. [2009], ‘Some asymptotic results on generalized penalized spline smoothing’, Journal of the Royal Statistical Society: Series B (Statistical Methodology) 71(2), 487–503.
  • [19] Krivobokova, T., Kneib, T. and Claeskens, G. [2010], ‘Simultaneous confidence bands for penalized spline estimators’, Journal of the American Statistical Association 105(490), 852–863.
  • [20] Lai, M.-J. and Wang, L. [2013], ‘Bivariate penalized splines for regression’, Statistica Sinica 23(3), 1399–1417.
  • [21] Li, Y. and Ruppert, D. [2008], ‘On the asymptotics of penalized splines’, Biometrika 95(2), 415–436.
  • [22] O’Sullivan, F. [1986], ‘A statistical perspective on ill-posed inverse problems’, Statist. Sci. 1(4), 502–518.
  • [23] Rudelson, M. and Vershynin, R. [2013], ‘Hanson-wright inequality and sub-gaussian concentration’, Electron. Commun. Probab. 18, 9 pp.
  • [24] Ruppert, D., Wand, M. and Carroll, R. [2003], Semiparametric Regression, Cambridge, New York.
  • [25] Ruppert, D., Wand, M. and Carroll, R. J. [2009], ‘Semiparametric regression during 2003?2007’, Electron. J. Statist. 3, 1193–1256.
  • [26] Schumaker, L. [1981], Spline Functions: Basic Theory, Wiley-Interscience.
  • [27] Schwarz, K. and Krivobokova, T. [2016], ‘A unified framework for spline estimators’, Biometrika 103(1), 121.
  • [28] Seber, G. [2007], A Matrix Handbook for Statisticians, Wiley-Interscience, New Jersey.
  • [29] Serfling, R. [1980], Approximation Theorems of Mathematical Statistics, Wiley, New York.
  • [30] Silverman, B. W. [1984], ‘Spline smoothing: The equivalent variable kernel method’, Ann. Statist. 12(3), 898–916.
  • [31] Speckman, P. [1985], ‘Spline smoothing and optimal rates of convergence in nonparametric regression models’, Ann. Statist. 13(3), 970–983.
  • [32] Stone, C. J. [1982], ‘Optimal global rates of convergence for nonparametric regression’, Ann. Statist. 10(4), 1040–1053.
  • [33] Sun, J. and Loader, C. R. [1994], ‘Simultaneous confidence bands for linear regression and smoothing’, Ann. Statist. 22(3), 1328–1345.
  • [34] Utreras, F. [1983], ‘Natural spline functions, their associated eigenvalue problem’, Numerische Mathematik 42(1), 107–117.
  • [35] Vershynin, R. [2012], Introduction to the non-asymptotic analysis of random matrices, Cambridge University Press, p. 210–268.
  • [36] Wahba, G. [1990], Spline Models for Observational Data, Society for Industrial and Applied Mathematics.
  • [37] Wang, X., Shen, J. and Ruppert, D. [2011], ‘On the asymptotics of penalized spline smoothing’, Electron. J. Statist. 5, 1–17.
  • [38] Wood, S. [2006], ‘Low-rank scale-invariant tensor product smooths for generalized additive mixed models’, Biometrics 62, 1025–1036.
  • [39] Xiao, L. [2018], ‘Asymptotics of bivariate penalised splines’, Journal of Nonparametric Statistics 0(0), 1–26.
  • [40] Xiao, L., Li, C., Checkley, W. and Crainiceanu, C. [2018], ‘Fast covariance estimation for sparse functional data’, Statistics and Computing 28(3), 511–522.
  • [41] Xiao, L., Li, Y., Apanasovich, T. and Ruppert, D. [2012], Local asymptotics of P-splines. Technical report. Available at, https://arxiv.org/abs/1201.0708.
  • [42] Xiao, L., Li, Y. and Ruppert, D. [2013], ‘Fast bivariate, P-splines: the sandwich smoother’, J. R. Statist. Soc. B 75, 577–599.
  • [43] Xiao, L., Zipunnikov, V., Ruppert, D. and Crainiceanu, C. [2016], ‘Fast covariance estimation for high-dimensional functional data’, Statistics and Computing 26(1), 409–421.
  • [44] Yao, F. and Lee, T. C. M. [2006], ‘Penalized spline models for functional principal component analysis’, Journal of the Royal Statistical Society: Series B (Statistical Methodology) 68(1), 3–25.
  • [45] Yoshida, T. and Naito, K. [2014], ‘Asymptotics for penalised splines in generalised additive models’, Journal of Nonparametric Statistics 26(2), 269–289.
  • [46] Zhou, S., Shen, X. and Wolfe, D. A. [1998], ‘Local asymptotics for regression splines and confidence regions’, The Annals of Statistics 26(5), 1760–1782.