Statistical Science

A General Projection Framework for Constrained Smoothing

E. Mammen, J. S. Marron, B. A. Turlach, and M. P. Wand

Full-text: Open access


There are a wide array of smoothing methods available for finding structure in data. A general framework is developed which shows that many of these can be viewed as a projection of the data, with respect to appropriate norms. The underlying vector space is an unusually large product space, which allows inclusion of a wide range of smoothers in our setup (including many methods not typically considered to be projections). We give several applications of this simple geometric interpretation of smoothing. A major payoff is the natural and computationally frugal incorporation of constraints. Our point of view also motivates new estimates and helps understand the finite sample and asymptotic behavior of these estimates.

Article information

Statist. Sci., Volume 16, Issue 3 (2001), 232-248.

First available in Project Euclid: 24 December 2001

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Kernel smoothing local polynomials smoothing splines constrained smoothing monotone smoothing additive models


Mammen, E.; Marron, J. S.; Turlach, B. A.; Wand, M. P. A General Projection Framework for Constrained Smoothing. Statist. Sci. 16 (2001), no. 3, 232--248. doi:10.1214/ss/1009213727.

Export citation


  • Barlow, R. E. and van Zwet, W. R. (1970). Asy mptotic properties of isotonic estimators for generalized failure rate function I. Strong consistency. In Nonparametric Techniques in Statistical Inference (M. L. Puri, ed.) 159-173. Cambridge Univ. Press.
  • Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and Brunk, H. D. (1972). Statistical Inference under Order Restrictions. Wiley, New York
  • Bowman, A. W. and Azzalini, A. (1997). Applied Smoothing Techniques for Data Analy sis: The Kernel Approach with S-Plus Illustrations. Oxford Univ. Press.
  • Cheng, K. F. and Lin, P. E. (1981). Nonparametric estimation of a regression function. Z. Wahrsch. Verw. Gebiete 57 223-233.
  • Chu, C. K. and Marron, J. S. (1991). Choosing a kernel regression estimator (with discussion). Statist. Sci. 6 404-436.
  • Cox, D. D. (1988). Approximation of method of regularization estimators. Ann. Statist. 16 694-712.
  • Dechevsky, L. and MacGibbon (1999). Asy mptotically minimax nonparametric function estimation with positivity constraints i. Unpublished manuscript.
  • Dechevsky, L., MacGibbon, B. and Penev, S. (2001). Numerical methods for asy mptotically minimax nonparametric function estimation with positivity constraints i. Sankhy¯a. To appear.
  • Delecroix, M., Simioni, M. and Thomas-Agnan, C. (1995). A shape constrained smoother: simulation study. Comput. Statist. 10 155-175.
  • Delecroix, M., Simioni, M. and Thomas-Agnan, C. (1996). Functional estimation under shape constraints. J. Nonparametr. Statist. 6 69-89.
  • Delecroix, M. and Thomas-Agnan, C. (2000). Spline and kernel smoothing under shape restrictions. In Smoothing and Regression: Approaches, Computation and Application (M. Schimek, ed.) 109-134. Wiley, New York.
  • den Hertog, D. (1994). Interior Point Approach to Linear, Quadratic and Convex Programming. Kluwer, Dordrecht.
  • Dierckx, P. (1980). An algorithm for cubic spline fitting with convexity constraints. Computing 24 349-371.
  • Dole, D. (1999). C0Sm0: A constrained scatterplot smoother for estimating convex monotone transformations. J. Bus. Econom. Statist. 17 444.
  • Dy kstra, R. L. (1983). An algorithm for restricted least squares regression. J. Amer. Statist. Assoc. 77 621-628.
  • Efromovich, S. (1999). Nonparametric Curve Estimation: Methods, Theory, and Applications. Springer, New York.
  • Elfing, T. and Andersson, L. E. (1988). An algorithm for computing constrained smoothing spline functions. Numer. Math. 52 583-595.
  • Eubank, R. L. (1999). Smoothing Splines and Nonparametric Regression, 2nd ed. Dekker, New York.
  • Fan, J. and Gijbels, I. (1996). Local Poly nomial Modelling and Its Application. Chapman and Hall/CRC, New York.
  • Fisher, N. I., Hall, P., Turlach, B. A. and Watson, G. S. (1997). On the estimation of a convex set from noisy data on its support function. J. Amer. Statist. Assoc. 92 84-91.
  • Fletcher, R. (1987). Practical Methods of Optimization, 2nd ed. Wiley, New York.
  • F¨oldes, A. and R´ev´esz, P. (1974). A general method for density estimation, Studia Sci. Math. Hungar. 9 81-92.
  • Friedman, J. H. and Tibshirani, R. (1984). The monotone smoothing of scatterplots. Technometrics 26 243-250.
  • Gay lord, C. K. and Ramirez, D. E. (1991). Monotone regression splines for smoothed bootstrapping. Comput. Statist. Quarterly 6 85-97.
  • Goldfarb, D. and Idnani, A. (1983). A numerically stable dual method for solving strictly convex quadratic programs. Math. Programming 27 1-33.
  • Green, P. J. and Silverman, B. W. (1994). Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach. Chapman and Hall/CRC, London.
  • Hall, P. and Tajvidi, N. (2000). Distribution and dependence function estimation for bivariate extreme-value distributions. Bernoulli 6 835-844.
  • H¨ardle, W. (1990). Applied Non-parametric Regression. Cambridge Univ. Press.
  • H¨ardle W. and Gasser, T. (1984). Robust nonparametric function fitting, J. Roy. Statist. Soc. Ser. B 46 42-51.
  • H¨ardle, W. and Mammen, E. (1993). Comparing nonparametric versus parametric regression fits. Ann. Statist. 21 1926- 1947.
  • Hart, J. D. (1997). Nonparametric Smoothing and Lack-of-Fit Tests. Springer, New York.
  • Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. Chapman and Hall/CRC, London.
  • He, X. and Ng, P. (1999). COBS: qualitatively constrained smoothing via linear programming. Comput. Statist. 14 315-337.
  • Irvine, L. D., Marin, S. P. and Smith, P. W. (1986). Constrained interpolation and smoothing. Constr. Approx. 2 129-151.
  • Kelly, C. and Rice, J. (1990). Monotone smoothing with application to dose-response curves and the assessment of sy nergism. Biometrics 46 1071-1085.
  • Loader, C. (1999). Local Regression and Likelihood, Statistics and Computing. Springer, New York.
  • Mammen, E., Linton, O. and Nielsen, J. (1999). The existence and asy mptotic properties of a backfitting projection algorithm under weak conditions. Ann. Statist. 27 1443-1490.
  • Mammen, E. and Marron, J. S. (1997). Mass recentered kernel smoothers. Biometrika 84 765-778. Mammen, E., Marron, J. S., Turlach, B. A. and Wand, M. P.
  • (2001). Monotone local poly nomial smoothers. Unpublished manuscript.
  • Mammen, E. and Thomas-Agnan, C. (1999). Smoothing splines and shape restrictions. Scand. J. Statist. 26 239-252.
  • McCormick, G. P. (1983). Nonlinear Programming: Theory, Algorithms and Applications. Wiley, New York.
  • Micchelli, C. A. and Utreras, F. I. (1988). Smoothing and interpolation in a convex subset of a Hilbert space. SIAM J. Sci. Statist. Comput. 9 728-746.
  • Mukerjee, H. (1988). Monotone nonparametric regression. Ann. Statist. 16 741-750.
  • M ¨uller, H. G. (1988). Nonparametric Regression Analy sis of Longitudinal Data. Springer, New York.
  • Nash, S. G. and Sofer, A. (1996). Linear and Nonlinear Programming. McGraw-Hill, New York.
  • Opsomer, J. D. (2000). Ay smptotic properties of backfitting estimators. J. Multivariate Anal. 73 166-179.
  • Opsomer, J. D. and Ruppert, D. (1997). Fitting a bivariate additive model by local poly nomial regression. Ann. Statist. 25 186-211.
  • Prince, J. L. and Willsky, A. S. (1990). Reconstructing convex sets from support line measurements. IEEE Trans. Pattern Anal. Machine Intelligence 12 377-389.
  • Ramsay, J. O. (1988). Monotone regression splines in action (with discussion). Statist. Sci. 3 425-461.
  • Ramsay, J. O. (1998). Estimating smooth monotone functions. J. Roy. Statist. Soc. Ser. B 60 365-375.
  • Ramsay, J. O. and Silverman, B. W. (1997). Functional Data Analy sis. Springer, New York.
  • Ratkowsky, D. A. (1983). Nonlinear Regression Modeling. Dekker, New York.
  • Robertson, T., Wright, F. T. and Dy kstra, R. L. (1988). Order Restricted Statistical Inference. Wiley, New York.
  • Rudin, W. (1987). Real and Complex Analy sis. McGraw-Hill, New York.
  • Santal ´o, L. A. (1976). Integral Geometry and Geometric Probability. Addison-Wesley, Reading, MA.
  • Schmidt, J. W. (1987). An unconstrained dual program for computing convex C1-spline approximants. Computing 39 133- 140.
  • Schmidt, J. W. and Scholz, I. (1990). A dual algorithm for convex-concave data smoothing by cubic C2-splines. Numer. Math. 57 333-350.
  • Schwetlick, H. and Kunert, V. (1993). Spline smoothing under constraints on derivatives. BIT 33 512-528.
  • Silverman, B. W. (1986). Density Estimation for Statistics and Data Analy sis. Chapman and Hall/CRC, London.
  • Silverman, B. W. and Wood, J. T. (1987). The nonparametric estimation of branching curves. J. Amer. Statist. Assoc. 82 551-558.
  • Simonoff, J. S. (1996). Smoothing Methods in Statistics. Springer, New York.
  • Speckman, P. L. (1988). Kernel smoothing in partial linear models. J. Roy. Statist. Soc. Ser. B 50 413-436.
  • Steer, B. T. and Hocking, R. A. (1985). The optimum timing of nitrogen application to irrigated sunflowers. In Proceedings of the Eleventh International Sunflower Conference 221-226. Asociaci´on Argetina de Girasol, Buenos Aires.
  • Stone, C. J., Hansen, M. H., Kooperberg, C. and Truong, Y. K. (1997). Poly nomial splines and their tensor products in extended linear modeling (with discussion). Ann. Statist. 25 1371-1424.
  • Tantiy aswasdikul, C. and Woodroofe, M. B. (1994). Isotonic smoothing splines under sequential designs. J. Statist. Plann. Inference 38 75-88.
  • Tsy bakov, A. B. (1986). Robust reconstruction of functions by the local-approximation method. Problems Inform. Transmission 22 133-146.
  • Turlach, B. A. (1997). Constrained smoothing splines revisited. Statistics Research Report SRR 008-97, Center for Math. and Its Applications, Australian National Univ. Canberra.
  • Utreras, F. I. (1985). Smoothing noisy data under monotonicity constraints: Existence, characterization and convergence rates, Numer. Math. 47 611-625.
  • van de Geer, S. (1990). Estimating a regression function. Ann. Statist. 18 907-924.
  • Villalobos, M. and Wahba, G. (1987). Inequality-constrained multivariate smoothing splines with application to the estimation of posterior probabilities. J. Amer. Statist. Assoc. 82 239-248.
  • Wahba, G. (1990). Spline Functions for Observational Data. SIAM, Philadelphia.
  • Walter, G. G. and Blum, J. (1979). Probability density estimation using delta sequences. Ann. Statist. 7 328-340.
  • Wand, M. P. and Jones, M. C. (1995). Kernel smoothing. Chapman and Hall/CRC, London.
  • Wright, F. T. (1982). Monotone regression estimates for grouped observations. Ann. Statist. 10 278-286.