The Annals of Statistics

Locally adaptive regression splines

Enno Mammen and Sara van de Geer

Full-text: Open access


Least squares penalized regression estimates with total variation penalties are considered. It is shown that these estimators are least squares splines with locally data adaptive placed knot points. The definition of these variable knot splines as minimizers of global functionals can be used to study their asymptotic properties. In particular, these results imply that the estimates adapt well to spatially inhomogeneous smoothness. We show rates of convergence in bounded variation function classes and discuss pointwise limiting distributions. An iterative algorithm based on stepwise addition and deletion of knot points is proposed and its consistency proved.

Article information

Ann. Statist., Volume 25, Number 1 (1997), 387-413.

First available in Project Euclid: 10 October 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation
Secondary: 62G20: Asymptotic properties 62G30: Order statistics; empirical distribution functions

Nonparametric curve estimation penalized least squares splines local adaptivity rates of convergence


Mammen, Enno; van de Geer, Sara. Locally adaptive regression splines. Ann. Statist. 25 (1997), no. 1, 387--413. doi:10.1214/aos/1034276635.

Export citation


  • BABENKO, K. I., ed. 1979. Theoretical Foundations and Construction of Numerical Algorithms Z. for the Problems of Mathematical physics. Nauka, Moscow. In Russian. Z.
  • BALL, K. and PAJOR, A. 1990. The entropy of convex bodies with ``few'' extreme points. In Geometry of Banach Spaces. Cambridge Univ. Press. Z.
  • BARLOW, R. E., BARTHOLOMEW, D. J., BREMNER, J. M. and BRUNK, H. D. 1972. Statistical Inference under Order Restrictions. Wiley, New York. Z.
  • BIRGE, L. 1987. Estimating a density under order restrictions: nonasy mptotic minimax risk. ´ Ann. Statist. 15 995 1012.Z.
  • BIRMAN, M. S. and SOLOMJAK, M. Z. 1967. Piecewise-poly nomial approximations of functions of the classes W. Math. USSR-Sb. 2 295 317. p Z. Z
  • BREIMAN, C. 1991. The method for estimating multivariate functions from noisy data with. discussion. Technometrics 33 125 160. Z.
  • BREIMAN, C., FRIEDMAN, J. H., OLSHEN, R. A. and STONE, C. J. 1984. Classification and Regression Trees. Wadsworth, Belmont, CA. Z.
  • DE BOOR, C. 1978. A Practical Guide to Splines. Springer, Berlin. Z.
  • DE BOOR, C. and FIX, G. 1973. Spline approximation by quasi interpolants. J. Approx. Theory 8 19 45. Z.
  • DONOHO, D. L. and JOHNSTONE, I. M. 1994. Minimax estimation via wavelet shrinkage. Biometrika 81 425 455. Z.
  • DONOHO, D. L., JOHNSTONE, I. M., KERKy ACHARIAN, G. and PICARD, D. 1995. Wavelet shrinkage: Z. asy mptopia? with discussion. J. Roy al Statist. Soc. Ser. B 57 301 369. Z. Z.
  • FRIEDMAN, J. H. 1991. Multivariate adaptive regression splines with discussion. Ann. Statist. 19 1 141. Z.
  • FRIEDMAN, J. H. and SILVERMAN, B. W. 1989. Flexible parsimonious smoothing and additive Z. modeling with discussion. Technometrics 31 3 39. Z.
  • GIJBELS, I. and MAMMEN, E. 1994. On local adaptivity of kernel estimates with plug-in local bandwidth selectors. Unpublished manuscript. Z.
  • GROENEBOOM, P. 1985. Estimating a monotone density. Proceedings of the Berkeley Conferences Z. in Honor of J. Ney man and J. Kiefer L. M. LeCam and R. A. Olshen, eds. 2 539 555. Z.
  • GROENEBOOM, P. 1989. Brownian motion with a parabolic drift and Airy functions. Probab. Theory Related Fields 81 79 109. Z.
  • IBRAGIMOV, I. A. and HASMINSKII, R. Z. 1980. On nonparametric estimation of regression. Soviet Math. Dokl. 21 810 814. Z.
  • KOENKER, R., NG, P. T. and PORTNOY, S. 1994. Quantile smoothing splines. Biometrika 81 673 680. Z.
  • KUNSCH, H. R. 1994. Robust priors for smoothing and image restoration. Ann. Inst. Statist. ¨ Math. 46 1 19. Z.
  • LEPSKI, O. V., MAMMEN, E. and SPOKOINY, V. G. 1994. Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selectors. Ann. Statist. To appear. Z.
  • LEURGANS, S. 1982. Asy mptotic distributions of slope-of-greatest-convex-minorant estimators. Ann. Statist. 10 287 296. Z.
  • MAMMEN, E. 1991. Nonparametric regression under qualitative smoothness assumptions. Ann. Statist. 19 741 759. Z.
  • NEMIROVSKII, A. S., POLy AK, B. T. and TSy BAKOV, A. B. 1984. Signal processing via the nonparametric maximum likelihood method. Problemy Peredachi Informatsii 20 29 46. Z.
  • NEMIROVSKII, A. S., POLy AK, B. T. and TSy BAKOV, A. B. 1985. Rate of convergence of nonparametric estimates of maximum-likelihood ty pe. Problemy Peredachi Informatsii 21 258 272. Z.
  • PORTNOY, S. 1997. Local asy mptotics for quantile smoothing splines. Ann. Statist. 25 414 434. Z.
  • STONE, C. J. 1982. Optimal rates of convergence for nonparametric regression. Ann. Statist. 10 1040 1053.
  • STONE, C. J. 1994. The use of poly nomial splines and their tensor products in multivariate Z. function estimation with discussion. Ann. Statist. 22 118 171. Z. TSIREL'SON, B. S. 1982. A geometric approach to a maximum likelihood estimation for infinite dimensional location I. Theory Probab. Appl. 27 411 418. Z. TSIREL'SON, B. S. 1985. A geometric approach to a maximum likelihood estimation for infinite dimensional location II. Theory Probab. Appl. 30 820 828. Z. TSIREL'SON, B. S. 1986. A geometric approach to a maximum likelihood estimation for infinite dimensional location III. Theory Probab. Appl. 31 470 483. Z.
  • VAN DE GEER, S. 1990. Estimating a regression function. Ann. Statist. 18 907 924. Z.
  • WRIGHT, F. T. 1981. The asy mptotic behavior of monotone regression estimates. Ann. Statist. 9 443 448.