The Annals of Statistics

Nonparametric kernel regression subject to monotonicity constraints

Peter Hall and Li-Shan Huang

Full-text: Open access


We suggest a method for monotonizing general kernel-type estimators, for example local linear estimators and Nadaraya .Watson estimators. Attributes of our approach include the fact that it produces smooth estimates, indeed with the same smoothness as the unconstrained estimate. The method is applicable to a particularly wide range of estimator types, it can be trivially modified to render an estimator strictly monotone and it can be employed after the smoothing step has been implemented. Therefore,an experimenter may use his or her favorite kernel estimator, and their favorite bandwidth selector, to construct the basic nonparametric smoother and then use our technique to render it monotone in a smooth way. Implementation involves only an off-the-shelf programming routine. The method is based on maximizing fidelity to the conventional empirical approach, subject to monotonicity.We adjust the unconstrained estimator by tilting the empirical distribution so as to make the least possible change, in the sense of a distance measure, subject to imposing the constraint of monotonicity.

Article information

Ann. Statist., Volume 29, Issue 3 (2001), 624-647.

First available in Project Euclid: 24 December 2001

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G07: Density estimation
Secondary: 62G20: Asymptotic properties

Bandwidth,,,,,,,,,. biased bootstrap Gasser –Muller estimator isotonic regression local linear estimator Nadaraya-Watson estimator order restricted inference power divergence Priestley –Chao estimator weighted bootstrap


Hall, Peter; Huang, Li-Shan. Nonparametric kernel regression subject to monotonicity constraints. Ann. Statist. 29 (2001), no. 3, 624--647. doi:10.1214/aos/1009210683.

Export citation


  • Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and Brunk, H. D. (1972). Statistical Inference Under Order Restrictions. Wiley, New York.
  • Bartholomew, D. J. (1959). A test of homogeneity for ordered alternatives. Biometrika 46 36-48.
  • Bloch, D. A. and Silverman, B. W. (1997). Monotone discriminant functions and their applications in rheumatology. J. Amer. Statist. Assoc. 92 144-153.
  • Bowman, A. W. and Azzalini, A. (1997). Applied Smoothing Techniques for Data Analysis. Oxford Univ. Press.
  • Bowman, A. W., Jones, M. C. and Gijbels, I. (1998). Testing monotonicity of regression. J. Comput. Graph. Statist. 7 489-500.
  • Brunk, H. D. (1955). Maximum likelihood estimates of monotone parameters. Ann. Math. Statist. 26 607-616.
  • Cressie, N. A. C. and Read, T. R. C. (1984). Multinomial goodness-of-fit tests. J. Roy. Statist. Soc. Ser. B 46 440-464.
  • Hall, P. and Heckman, N. (2000). Testing for monotonicity of a regression mean by calibrating for linear functionals. Ann Statist. 28 20-39.
  • Hall, P. and Presnell, B. (1999). Intentionally biased bootstrap methods. J. Roy. Statist. Soc. Ser. B 61 143-158.
  • Fisher, N. I., Hall, P., Turlach, B. A. and Watson, G. S. (1997). On the estimation of a convex set from noisy data on its support function. J. Amer. Statist. Assoc. 92 84-91.
  • Friedman, J. H. and Tibshirani, R. J. (1984). The monotone smoothing of scatterplots. Technometrics 26 243-250.
  • Gonz´alez-Manteiga, W., Cao, R. and Marron, J. S. (1996). Bootstrap selection of the smoothing parameter in nonparametric hazard rate estimation. J. Amer. Statist. Assoc. 91 1130-1140.
  • Hougaard, P. (1988). A boundary modification of kernel function smoothing, with application to insulin absorption kinetics. In Compstat Lectures 31-36. Physica, Vienna.
  • Hougaard, P., Plum, A. and Ribel, U. (1989). Kernel function smoothing of insulin absorption kinetics. Biometrics 45 1041-1052.
  • Kelly, C. and Rice, J. (1990). Monotone smoothing with application to dose-response curves and the assessment of synergism. Biometrics 46 1071-1085.
  • Mammen, E. (1991). Estimating a smooth regression function. Ann. Statist. 19 724-740.
  • Mammen, E. and Thomas-Agnan, C. (1999). Smoothing splines and shape restrictions. Scand. J. Statist. 26 239-252.
  • Mammen, E. and Marron, J. S. (1997). Mass centred kernel smoothers. Biometrika 84 765-777. Mammen, E., Marron, J. S., Turlach, B. A. and Wand, M. P. (1999a). A general framework for constrained smoothing. Unpublished manuscript. Mammen, E., Marron, J. S., Turlach, B. A. and Wand, M. P. (1999b). Monotone local polynomial smoothers. Unpublished manuscript.
  • Marron, J. S., Turlach, B. A. and Wand, M. P. (1997). Local polynomial smoothing under qualitative constraints. In Graph-Image-Vision (L. Billard and N. I. Fisher, eds.) 647-652. Interface Foundation of North America, Fairfax Station, VA.
  • Mukerjee, H. (1988). Monotone nonparametric regression. Ann. Statist. 16 741-750.
  • M ¨uller, H.-G. (1997). Density adjusted kernel smoothers for random design nonparametric regression. Statist. Probab. Lett. 36 161-172.
  • M ¨uller, H.-G. and Song, K.-S. (1993). Identity reproducing multivariate nonparametric regression. J. Multivariate Anal. 46 237-253.
  • Patil, P., Wells, M. T. and Marron, J. S. (1994). Some heuristics of kernel based estimators of ratio functions. J. Nonparametr. Statist. 4 203-209.
  • Pearson, G. W. and Qua, F. (1993). High precision14C measurement of Irish oaks to show the natural14 variations from AD 1840-5000 BC: a correction. Radiocarbon 35 105-123. Poiner, I. R., Blaber, S. J. M., Brewer, D. T., Burridge, C. Y., Caesar, D., Connell, M., Dennis, D., Dews, G. D., Ellis, A. N., Farmer, M., Fry, G. J., Glaister, J., Gribble, N., Hill, B. J., Long, B. G., Milton, D. A., Pitcher, C. R., Proh, D., Salini, J. P.,
  • Thomas, M. R., Toscas, P., Veronise, S., Wang, Y. G. and Wassenberg, T. J. (1997). The effects of prawn trawling in the far northern section of the Great Barrier Reef. Final report to CBRMPA and FRDC on 1991-96 research. CSIRO Division of Marine Research, Queensland Dept. Primary Industries.
  • Ramsay, J. O. (1988). Monotone regression splines in action (with comments). Statist. Sci. 3 425-461.
  • Ramsay, J. O. (1998). Estimating smooth monotone functions. J. Roy. Statist. Soc. Ser. B 60 365-375.
  • Schlee, W. (1982). Nonparametric tests of the monotony and convexity of regression. In Nonparametric Statistical Inference II (B. V. Gnedenko, M. L. Puri and I. Vincze, eds.) 823-836. North-Holland, Amsterdam.
  • Turlach, B. A. (1997). Constrained smoothing splines revisited. Technical report SSR97-008, Australian National Univ., Centre for Mathematics and Its Applications.
  • Wand, M. P. and Jones, M. C. (1995). Kernel Smoothing. Chapman and Hall, London.