Electronic Journal of Statistics

The bias of isotonic regression

Ran Dai, Hyebin Song, Rina Foygel Barber, and Garvesh Raskutti

Full-text: Open access


We study the bias of the isotonic regression estimator. While there is extensive work characterizing the mean squared error of the isotonic regression estimator, relatively little is known about the bias. In this paper, we provide a sharp characterization, proving that the bias scales as $O(n^{-\beta /3})$ up to log factors, where $1\leq \beta \leq 2$ is the exponent corresponding to Hölder smoothness of the underlying mean. Importantly, this result only requires a strictly monotone mean and that the noise distribution has subexponential tails, without relying on symmetric noise or other restrictive assumptions.

Article information

Electron. J. Statist., Volume 14, Number 1 (2020), 801-834.

Received: August 2019
First available in Project Euclid: 5 February 2020

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression

Isotonic regression bias

Creative Commons Attribution 4.0 International License.


Dai, Ran; Song, Hyebin; Barber, Rina Foygel; Raskutti, Garvesh. The bias of isotonic regression. Electron. J. Statist. 14 (2020), no. 1, 801--834. doi:10.1214/20-EJS1677. https://projecteuclid.org/euclid.ejs/1580871776

Export citation


  • E. S. Andersen. On the fluctuations of sums of random variables ii., Mathematica Scandinavica, 2:194–222, Dec. 1954. URL https://www.mscand.dk/article/view/10407.
  • M. Banerjee, C. Durot, and B. Sen. Divide and conquer in nonstandard problems and the super-efficiency phenomenon., Ann. Statist., 47(2):720–757, 2019. ISSN 0090-5364. URL https://doi.org/10.1214/17-AOS1633.
  • R. E. Barlow, D. J. Bartholomew, J. Bremner, and H. Brunk. Statistical inference under order restrictions: The theory and application of isotonic regression., Wiley New York, 1972.
  • D. J. Bartholomew. A test for homogeneity for ordered alternatives., Biometrika, 46:36–48, 1959a.
  • D. J. Bartholomew. A test for homogeneity for ordered alternatives ii., Biometrika, 46:328–335, 1959b.
  • H. D. Brunk. Maximum likelihood estimates of monotone parameters., Annals of Mathematical Statistics, 26:607–616, 1955.
  • H. D. Brunk. Estimation of isotonic regression. In, Nonparametric Techniques in Statistical Inference (Proc. Sympos., Indiana Univ., Bloomington, Ind., 1969), pages 177–197. Cambridge Univ. Press, London, 1970.
  • E. Cator. Adaptivity and optimality of the monotone least-squares estimator., Bernoulli, 17(2):714–735, 05 2011. URL https://doi.org/10.3150/10-BEJ289.
  • S. Chatterjee, A. Guntuboyina, and B. Sen. On risk bounds in isotonic and other shape restricted regression problems., The Annals of Statistics, 43 :1774–1800, 08 2015.
  • J. de Leeuw, K. Hornik, and P. Mair. Isotone optimization in r: Pool-adjacent-violators (pava) and active set methods., Journal of Statistical Software, 32:1–24, 2009.
  • C. Durot. Sharp asymptotics for isotonic regression., Probability Theory and Related Fields, 122(2):222–240, Feb 2002. ISSN 1432-2064. URL https://doi.org/10.1007/s004400100171.
  • C. Durot, V. N. Kulikov, and H. P. Lopuhaä. The limit distribution of the $l_\infty $-error of grenander-type estimators., Ann. Statist., 40(3) :1578–1608, 06 2012. URL https://doi.org/10.1214/12-AOS1015.
  • C. Gao, F. Han, and C.-H. Zhang. On estimation of isotonic piecewise constant signals., arXiv preprint arXiv:1705.06386, 2017.
  • U. Grenander. On the theory of mortality measurement: part ii., Scandinavian Actuarial Journal, 1956(2):125–153, 1956.
  • A. Guntuboyina and B. Sen. Nonparametric shape-restricted regression., Statistical Science, 33:568–594, 2018.
  • Q. Han, T. Wang, S. Chatterjee, and R. J. Samworth. Isotonic regression in general dimensions., ArXiv e-prints, 2017.
  • B. Laurent and P. Massart. Adaptive estimation of a quadratic functional by model selection., Annals of Statistics, pages 1302–1338, 2000.
  • R. E. Miles. The complete amalgamation into blocks, by weighted means, of a finite set of real numbers., Biometrika, 46:317–327, 1959.
  • T. Robertson, F. Wright, and R. Dykstra., Order Restricted Statistical Inference. Probability and Statistics Series. Wiley, 1988. ISBN 9780471917878. URL https://books.google.com/books?id=sqZfQgAACAAJ.
  • A. I. Sakhanenko. Convergence rate in the invariance principle for non-identically distributed variables with exponential moments., Advances in Probability Theory: Limit Theorems for Sums of Random Variables, pages 2–73, 1985.
  • F. T. Wright. The asymptotic behavior of monotone regression estimates., Ann. Statist., 9(2):443–448, 1981. ISSN 0090-5364.
  • F. Yang and R. F. Barber. Contraction and uniform convergence of isotonic regression., ArXiv e-prints, 2018.