Statistical Science

Limit Theory in Monotone Function Estimation

Cécile Durot and Hendrik P. Lopuhaä

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


We give an overview of the different concepts and methods that are commonly used when studying the asymptotic properties of isotonic estimators. After introducing the inverse process, we illustrate its use in establishing weak convergence of the estimators at a fixed point and also weak convergence of global distances, such as the $\mathbb{L}_{p}$-distance and supremum distance. Furthermore, we discuss the developments on smooth isotonic estimation.

Article information

Statist. Sci., Volume 33, Number 4 (2018), 547-567.

First available in Project Euclid: 29 November 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Cox model current status model isotonic estimation limit theory $\mathbb{L}_{p}$-distance maximum likelihood estimators monotone density monotone failure rate monotone regression supremum distance


Durot, Cécile; Lopuhaä, Hendrik P. Limit Theory in Monotone Function Estimation. Statist. Sci. 33 (2018), no. 4, 547--567. doi:10.1214/18-STS664.

Export citation


  • [1] Akakpo, N., Balabdaoui, F. and Durot, C. (2014). Testing monotonicity via local least concave majorants. Bernoulli 20 514–544.
  • [2] Anevski, D. and Hössjer, O. (2002). Monotone regression and density function estimation at a point of discontinuity. J. Nonparametr. Stat. 14 279–294.
  • [3] Anevski, D. and Hössjer, O. (2006). A general asymptotic scheme for inference under order restrictions. Ann. Statist. 34 1874–1930.
  • [4] Anevski, D. and Soulier, P. (2011). Monotone spectral density estimation. Ann. Statist. 39 418–438.
  • [5] Bagchi, P., Banerjee, M. and Stoev, S. A. (2016). Inference for monotone functions under short- and long-range dependence: Confidence intervals and new universal limits. J. Amer. Statist. Assoc. 111 1634–1647.
  • [6] Balabdaoui, F., Durot, C. and Jankowski, H. (2016). Least squares estimation in the monotone single index model. Preprint. Available at arXiv:1610.06026.
  • [7] Balabdaoui, F., Jankowski, H., Pavlides, M., Seregin, A. and Wellner, J. (2011). On the Grenander estimator at zero. Statist. Sinica 21 873–899.
  • [8] Banerjee, M., Durot, C. and Sen, B. (2016). Divide and conquer in non-standard problems and the super-efficiency phenomenon. Preprint. Available at arXiv:1605.04446.
  • [9] Banerjee, M. and Wellner, J. A. (2001). Likelihood ratio tests for monotone functions. Ann. Statist. 29 1699–1731.
  • [10] Banerjee, M. and Wellner, J. A. (2005). Confidence intervals for current status data. Scand. J. Stat. 32 405–424.
  • [11] Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and Brunk, H. D. (1972). Statistical Inference Under Order Restrictions. The Theory and Application of Isotonic Regression. Wiley, New York.
  • [12] Beare, B. K. and Schmidt, L. D. W. (2016). An empirical test of pricing kernel monotonicity. J. Appl. Econometrics 31 338–356.
  • [13] Bellec, P. C. (2018). Optimal bounds for aggregation of affine estimators. Ann. Statist. 46 30–59.
  • [14] Brunk, H. D. (1955). Maximum likelihood estimates of monotone parameters. Ann. Math. Stat. 26 607–616.
  • [15] Carolan, C. and Dykstra, R. (1999). Asymptotic behavior of the Grenander estimator at density flat regions. Canad. J. Statist. 27 557–566.
  • [16] Carroll, R. J., Delaigle, A. and Hall, P. (2011). Testing and estimating shape-constrained nonparametric density and regression in the presence of measurement error. J. Amer. Statist. Assoc. 106 191–202.
  • [17] Chatterjee, S., Guntuboyina, A. and Sen, B. (2015). On risk bounds in isotonic and other shape restricted regression problems. Ann. Statist. 43 1774–1800.
  • [18] Chatterjee, S., Guntuboyina, A. and Sen, B. (2018). On matrix estimation under monotonicity constraints. Bernoulli 24 1072–1100.
  • [19] Cheng, K. F. and Lin, P. E. (1981). Nonparametric estimation of a regression function. Z. Wahrsch. Verw. Gebiete 57 223–233.
  • [20] Chernoff, H. (1964). Estimation of the mode. Ann. Inst. Statist. Math. 16 31–41.
  • [21] Dedecker, J., Merlevède, F. and Peligrad, M. (2011). Invariance principles for linear processes with application to isotonic regression. Bernoulli 17 88–113.
  • [22] Du, P., Parmeter, C. F. and Racine, J. S. (2013). Nonparametric kernel regression with multiple predictors and multiple shape constraints. Statist. Sinica 23 1347–1371.
  • [23] Durot, C. (2002). Sharp asymptotics for isotonic regression. Probab. Theory Related Fields 122 222–240.
  • [24] Durot, C. (2003). A Kolmogorov-type test for monotonicity of regression. Statist. Probab. Lett. 63 425–433.
  • [25] Durot, C. (2007). On the $\mathbb{L}_{p}$-error of monotonicity constrained estimators. Ann. Statist. 35 1080–1104.
  • [26] Durot, C. (2008). Monotone nonparametric regression with random design. Math. Methods Statist. 17 327–341.
  • [27] Durot, C. (2008). Testing convexity or concavity of a cumulated hazard rate. IEEE Trans. Reliab. 57 465–473.
  • [28] Durot, C., Groeneboom, P. and Lopuhaä, H. P. (2013). Testing equality of functions under monotonicity constraints. J. Nonparametr. Stat. 25 939–970.
  • [29] Durot, C., Kulikov, V. N. and Lopuhaä, H. P. (2012). The limit distribution of the $L_{\infty}$-error of Grenander-type estimators. Ann. Statist. 40 1578–1608.
  • [30] Durot, C. and Lopuhaä, H. P. (2014). A Kiefer–Wolfowitz type of result in a general setting, with an application to smooth monotone estimation. Electron. J. Stat. 8 2479–2513.
  • [31] Durot, C. and Reboul, L. (2010). Goodness-of-fit test for monotone functions. Scand. J. Stat. 37 422–441.
  • [32] Durot, C. and Tocquet, A.-S. (1998). Goodness of fit test for isotonic regression. C. R. Acad. Sci. Paris Sér. I Math. 327 199–204.
  • [33] Durot, C. and Tocquet, A.-S. (2001). Goodness of fit test for isotonic regression. ESAIM Probab. Stat. 5 119–140.
  • [34] Durot, C. and Tocquet, A.-S. (2003). On the distance between the empirical process and its concave majorant in a monotone regression framework. Ann. Inst. Henri Poincaré Probab. Stat. 39 217–240.
  • [35] Eggermont, P. P. B. and LaRiccia, V. N. (2000). Maximum likelihood estimation of smooth monotone and unimodal densities. Ann. Statist. 28 922–947.
  • [36] Friedman, J. and Tibshirani, R. (1984). The monotone smoothing of scatter plots. Technometrics 26 243–350.
  • [37] Grenander, U. (1956). On the theory of mortality measurement. II. Skand. Aktuarietidskr. 39 125–153.
  • [38] Groeneboom, P. (1983). The concave majorant of Brownian motion. Ann. Probab. 11 1016–1027.
  • [39] Groeneboom, P. (1984). Brownian Motion With a Parabolic Drift and Airy Functions. CWI technical report, Dept. Mathematical Statistics-R 8413, CWI, Amsterdam.
  • [40] Groeneboom, P. (1985). Estimating a monotone density. In Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, Vol. II (Berkeley, Calif., 1983). 539–555. Wadsworth, Belmont, CA.
  • [41] Groeneboom, P. (1989). Brownian motion with a parabolic drift and Airy functions. Probab. Theory Related Fields 81 79–109.
  • [42] Groeneboom, P. (2013). Nonparametric (smoothed) likelihood and integral equations. J. Statist. Plann. Inference 143 2039–2065.
  • [43] Groeneboom, P. and Hendrickx, K. (2018). Current status linear regression. Ann. Statist. 46 1415–1444.
  • [44] Groeneboom, P., Hooghiemstra, G. and Lopuhaä, H. P. (1999). Asymptotic normality of the $L_{1}$ error of the Grenander estimator. Ann. Statist. 27 1316–1347.
  • [45] Groeneboom, P. and Jongbloed, G. (2010). Generalized continuous isotonic regression. Statist. Probab. Lett. 80 248–253.
  • [46] Groeneboom, P. and Jongbloed, G. (2013). Smooth and non-smooth estimates of a monotone hazard. In From Probability to Statistics and Back: High-Dimensional Models and Processes. Inst. Math. Stat. (IMS) Collect. 9 174–196. IMS, Beachwood, OH.
  • [47] Groeneboom, P. and Jongbloed, G. (2013). Testing monotonicity of a hazard: Asymptotic distribution theory. Bernoulli 19 1965–1999.
  • [48] Groeneboom, P. and Jongbloed, G. (2014). Nonparametric Estimation Under Shape Constraints: Estimators, Algorithms and Asymptotics. Cambridge Series in Statistical and Probabilistic Mathematics 38. Cambridge Univ. Press, New York.
  • [49] Groeneboom, P. and Jongbloed, G. (2015). Nonparametric confidence intervals for monotone functions. Ann. Statist. 43 2019–2054.
  • [50] Groeneboom, P., Jongbloed, G. and Witte, B. I. (2010). Maximum smoothed likelihood estimation and smoothed maximum likelihood estimation in the current status model. Ann. Statist. 38 352–387.
  • [51] Groeneboom, P. and Lopuhaä, H. P. (1993). Isotonic estimators of monotone densities and distribution functions: Basic facts. Stat. Neerl. 47 175–183.
  • [52] Groeneboom, P. and Wellner, J. A. (1992). Information Bounds and Nonparametric Maximum Likelihood Estimation. DMV Seminar 19. Birkhäuser, Basel.
  • [53] Groeneboom, P. and Wellner, J. A. (2001). Computing Chernoff’s distribution. J. Comput. Graph. Statist. 10 388–400.
  • [54] Han, Q., Wang, T., Chatterjee, S. and Samworth, R. J. (2017). Isotonic regression in general dimensions. Preprint. Available at arXiv:1708.09468.
  • [55] Hooghiemstra, G. and Lopuhaä, H. P. (1998). An extremal limit theorem for the argmax process of Brownian motion minus a parabolic drift. Extremes 1 215–240.
  • [56] Huang, J. and Wellner, J. A. (1995). Estimation of a monotone density or monotone hazard under random censoring. Scand. J. Stat. 22 3–33.
  • [57] Huang, Y. and Zhang, C.-H. (1994). Estimating a monotone density from censored observations. Ann. Statist. 22 1256–1274.
  • [58] Jankowski, H. (2014). Convergence of linear functionals of the Grenander estimator under misspecification. Ann. Statist. 42 625–653.
  • [59] Jankowski, H. K. and Wellner, J. A. (2009). Estimation of a discrete monotone distribution. Electron. J. Stat. 3 1567–1605.
  • [60] Jonker, M. A. and van der Vaart, A. W. (2001). A semi-parametric model for censored and passively registered data. Bernoulli 7 1–31.
  • [61] Kiefer, J. and Wolfowitz, J. (1976). Asymptotically minimax estimation of concave and convex distribution functions. Z. Wahrsch. Verw. Gebiete 34 73–85.
  • [62] Kim, J. and Pollard, D. (1990). Cube root asymptotics. Ann. Statist. 18 191–219.
  • [63] Komlós, J., Major, P. and Tusnády, G. (1975). An approximation of partial sums of independent $\mathrm{RV}$’s and the sample $\mathrm{DF}$. I. Z. Wahrsch. Verw. Gebiete 32 111–131.
  • [64] Kosorok, M. R. (2008). Bootstrapping in Grenander estimator. In Beyond Parametrics in Interdisciplinary Research: Festschrift in Honor of Professor Pranab K. Sen. Inst. Math. Stat. (IMS) Collect. 1 282–292. IMS, Beachwood, OH.
  • [65] Kulikov, V. N. and Lopuhaä, H. P. (2005). Asymptotic normality of the $L_{k}$-error of the Grenander estimator. Ann. Statist. 33 2228–2255.
  • [66] Kulikov, V. N. and Lopuhaä, H. P. (2006). The behavior of the NPMLE of a decreasing density near the boundaries of the support. Ann. Statist. 34 742–768.
  • [67] Kulikov, V. N. and Lopuhaä, H. P. (2006). The limit process of the difference between the empirical distribution function and its concave majorant. Statist. Probab. Lett. 76 1781–1786.
  • [68] Kulikov, V. N. and Lopuhaä, H. P. (2008). Distribution of global measures of deviation between the empirical distribution function and its concave majorant. J. Theoret. Probab. 21 356–377.
  • [69] Leurgans, S. (1982). Asymptotic distributions of slope-of-greatest-convex-minorant estimators. Ann. Statist. 10 287–296.
  • [70] Lopuhaä, H. P. and Musta, E. (2016). A central limit theorem for the Hellinger loss of Grenander type estimators. Available at arXiv:1612.06647.
  • [71] Lopuhaä, H. P. and Musta, E. (2017). Isotonized smooth estimators of a monotone baseline hazard in the Cox model. J. Statist. Plann. Inference 191 43–67.
  • [72] Lopuhaä, H. P. and Musta, E. (2017). Smooth estimation of a monotone hazard and a monotone density under random censoring. Stat. Neerl. 71 58–82.
  • [73] Lopuhaä, H. P. and Musta, E. (2018). Smoothed isotonized estimators of a monotone baseline hazard in the Cox model. Scand. J. Stat. 45 753–791.
  • [74] Lopuhaä, H. P. and Musta, E. (2018). The distance between a naive cumulative estimator and its least concave majorant. Statist. Probab. Lett. 139 119–128.
  • [75] Lopuhaä, H. P. and Nane, G. F. (2013). Shape constrained non-parametric estimators of the baseline distribution in Cox proportional hazards model. Scand. J. Stat. 40 619–646.
  • [76] Mammen, E. (1991). Estimating a smooth monotone regression function. Ann. Statist. 19 724–740.
  • [77] Marshall, A. W. and Proschan, F. (1965). Maximum likelihood estimation for distributions with monotone failure rate. Ann. Math. Stat. 36 69–77.
  • [78] Meyer, M. C. (2008). Inference using shape-restricted regression splines. Ann. Appl. Stat. 2 1013–1033.
  • [79] Mukerjee, H. (1988). Monotone nonparameteric regression. Ann. Statist. 16 741–750.
  • [80] Murphy, S. A., van der Vaart, A. W. and Wellner, J. A. (1999). Current status regression. Math. Methods Statist. 8 407–425.
  • [81] Nane, G. F. (2015). A likelihood ratio test for monotone baseline hazard functions in the Cox model. Statist. Sinica 25 1163–1184.
  • [82] Pal, J. K. and Woodroofe, M. (2006). On the distance between cumulative sum diagram and its greatest convex minorant for unequally spaced design points. Scand. J. Stat. 33 279–291.
  • [83] Prakasa Rao, B. L. S. (1969). Estkmation of a unimodal density. Sankhyā Ser. A 31 23–36.
  • [84] Prakasa Rao, B. L. S. (1970). Estimation for distributions with monotone failure rate. Ann. Math. Stat. 41 507–519.
  • [85] Ramsay, J. O. (1998). Estimating smooth monotone functions. J. R. Stat. Soc. Ser. B. Stat. Methodol. 60 365–375.
  • [86] Sen, B., Banerjee, M. and Woodroofe, M. (2010). Inconsistency of bootstrap: The Grenander estimator. Ann. Statist. 38 1953–1977.
  • [87] Söhl, J. (2015). Uniform central limit theorems for the Grenander estimator. Electron. J. Stat. 9 1404–1423.
  • [88] Sun, J. and Woodroofe, M. (1996). Adaptive smoothing for a penalized NPMLE of a non-increasing density. J. Statist. Plann. Inference 52 143–159.
  • [89] Tantiyaswasdikul, C. and Woodroofe, M. B. (1994). Isotonic smoothing splines under sequential designs. J. Statist. Plann. Inference 38 75–87.
  • [90] van der Vaart, A. W. and van der Laan, M. J. (2003). Smooth estimation of a monotone density. Statistics 37 189–203.
  • [91] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
  • [92] Wellner, J. A. (2015). Musings about shape constrained estimation and inference: Some problems. Presentation at workshop Shape Constrained Inference: Open Problems and New Directions, Lorentz Centre, Leiden, The Netherlands.
  • [93] Wellner, J. A. and Zhang, Y. (2000). Two estimators of the mean of a counting process with panel count data. Ann. Statist. 28 779–814.
  • [94] Woodroofe, M. and Sun, J. (1993). A penalized maximum likelihood estimate of $f(0+)$ when $f$ is nonincreasing. Statist. Sinica 3 501–515.
  • [95] Wright, F. T. (1981). The asymptotic behavior of monotone regression estimates. Ann. Statist. 9 443–448.
  • [96] Wright, F. T. (1982). Monotone regression estimates for grouped observations. Ann. Statist. 10 278–286.
  • [97] Zhao, O. and Woodroofe, M. (2012). Estimating a monotone trend. Statist. Sinica 22 359–378.