Electronic Journal of Statistics

The bias of isotonic regression

Ran Dai, Hyebin Song, Rina Foygel Barber, and Garvesh Raskutti

Full-text: Open access

Abstract

We study the bias of the isotonic regression estimator. While there is extensive work characterizing the mean squared error of the isotonic regression estimator, relatively little is known about the bias. In this paper, we provide a sharp characterization, proving that the bias scales as $O(n^{-\beta /3})$ up to log factors, where $1\leq \beta \leq 2$ is the exponent corresponding to Hölder smoothness of the underlying mean. Importantly, this result only requires a strictly monotone mean and that the noise distribution has subexponential tails, without relying on symmetric noise or other restrictive assumptions.

Article information

Source
Electron. J. Statist., Volume 14, Number 1 (2020), 801-834.

Dates
Received: August 2019
First available in Project Euclid: 5 February 2020

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1580871776

Digital Object Identifier
doi:10.1214/20-EJS1677

Mathematical Reviews number (MathSciNet)
MR4059933

Zentralblatt MATH identifier
07163274

Subjects
Primary: 62G08: Nonparametric regression

Keywords
Isotonic regression bias

Rights
Creative Commons Attribution 4.0 International License.

Citation

Dai, Ran; Song, Hyebin; Barber, Rina Foygel; Raskutti, Garvesh. The bias of isotonic regression. Electron. J. Statist. 14 (2020), no. 1, 801--834. doi:10.1214/20-EJS1677. https://projecteuclid.org/euclid.ejs/1580871776


Export citation

References

  • E. S. Andersen. On the fluctuations of sums of random variables ii., Mathematica Scandinavica, 2:194–222, Dec. 1954. URL https://www.mscand.dk/article/view/10407.
  • M. Banerjee, C. Durot, and B. Sen. Divide and conquer in nonstandard problems and the super-efficiency phenomenon., Ann. Statist., 47(2):720–757, 2019. ISSN 0090-5364. URL https://doi.org/10.1214/17-AOS1633.
  • R. E. Barlow, D. J. Bartholomew, J. Bremner, and H. Brunk. Statistical inference under order restrictions: The theory and application of isotonic regression., Wiley New York, 1972.
  • D. J. Bartholomew. A test for homogeneity for ordered alternatives., Biometrika, 46:36–48, 1959a.
  • D. J. Bartholomew. A test for homogeneity for ordered alternatives ii., Biometrika, 46:328–335, 1959b.
  • H. D. Brunk. Maximum likelihood estimates of monotone parameters., Annals of Mathematical Statistics, 26:607–616, 1955.
  • H. D. Brunk. Estimation of isotonic regression. In, Nonparametric Techniques in Statistical Inference (Proc. Sympos., Indiana Univ., Bloomington, Ind., 1969), pages 177–197. Cambridge Univ. Press, London, 1970.
  • E. Cator. Adaptivity and optimality of the monotone least-squares estimator., Bernoulli, 17(2):714–735, 05 2011. URL https://doi.org/10.3150/10-BEJ289.
  • S. Chatterjee, A. Guntuboyina, and B. Sen. On risk bounds in isotonic and other shape restricted regression problems., The Annals of Statistics, 43 :1774–1800, 08 2015.
  • J. de Leeuw, K. Hornik, and P. Mair. Isotone optimization in r: Pool-adjacent-violators (pava) and active set methods., Journal of Statistical Software, 32:1–24, 2009.
  • C. Durot. Sharp asymptotics for isotonic regression., Probability Theory and Related Fields, 122(2):222–240, Feb 2002. ISSN 1432-2064. URL https://doi.org/10.1007/s004400100171.
  • C. Durot, V. N. Kulikov, and H. P. Lopuhaä. The limit distribution of the $l_\infty $-error of grenander-type estimators., Ann. Statist., 40(3) :1578–1608, 06 2012. URL https://doi.org/10.1214/12-AOS1015.
  • C. Gao, F. Han, and C.-H. Zhang. On estimation of isotonic piecewise constant signals., arXiv preprint arXiv:1705.06386, 2017.
  • U. Grenander. On the theory of mortality measurement: part ii., Scandinavian Actuarial Journal, 1956(2):125–153, 1956.
  • A. Guntuboyina and B. Sen. Nonparametric shape-restricted regression., Statistical Science, 33:568–594, 2018.
  • Q. Han, T. Wang, S. Chatterjee, and R. J. Samworth. Isotonic regression in general dimensions., ArXiv e-prints, 2017.
  • B. Laurent and P. Massart. Adaptive estimation of a quadratic functional by model selection., Annals of Statistics, pages 1302–1338, 2000.
  • R. E. Miles. The complete amalgamation into blocks, by weighted means, of a finite set of real numbers., Biometrika, 46:317–327, 1959.
  • T. Robertson, F. Wright, and R. Dykstra., Order Restricted Statistical Inference. Probability and Statistics Series. Wiley, 1988. ISBN 9780471917878. URL https://books.google.com/books?id=sqZfQgAACAAJ.
  • A. I. Sakhanenko. Convergence rate in the invariance principle for non-identically distributed variables with exponential moments., Advances in Probability Theory: Limit Theorems for Sums of Random Variables, pages 2–73, 1985.
  • F. T. Wright. The asymptotic behavior of monotone regression estimates., Ann. Statist., 9(2):443–448, 1981. ISSN 0090-5364.
  • F. Yang and R. F. Barber. Contraction and uniform convergence of isotonic regression., ArXiv e-prints, 2018.