Annals of Statistics

On estimation of isotonic piecewise constant signals

Chao Gao, Fang Han, and Cun-Hui Zhang

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


Consider a sequence of real data points $X_{1},\ldots ,X_{n}$ with underlying means $\theta ^{*}_{1},\dots ,\theta ^{*}_{n}$. This paper starts from studying the setting that $\theta ^{*}_{i}$ is both piecewise constant and monotone as a function of the index $i$. For this, we establish the exact minimax rate of estimating such monotone functions, and thus give a nontrivial answer to an open problem in the shape-constrained analysis literature. The minimax rate under the loss of the sum of squared errors involves an interesting iterated logarithmic dependence on the dimension, a phenomenon that is revealed through characterizing the interplay between the isotonic shape constraint and model selection complexity. We then develop a penalized least-squares procedure for estimating the vector $\theta ^{*}=(\theta^{*}_{1},\dots ,\theta ^{*}_{n})^{\mathsf{T}}$. This estimator is shown to achieve the derived minimax rate adaptively. For the proposed estimator, we further allow the model to be misspecified and derive oracle inequalities with the optimal rates, and show there exists a computationally efficient algorithm to compute the exact solution.

Article information

Ann. Statist., Volume 48, Number 2 (2020), 629-654.

Received: July 2018
Revised: November 2018
First available in Project Euclid: 26 May 2020

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Primary: 62G08: Nonparametric regression 62C20: Minimax procedures

Isotonic piecewise constant function reduced isotonic regression iterated logarithmic dependence adaptive estimation oracle inequalities


Gao, Chao; Han, Fang; Zhang, Cun-Hui. On estimation of isotonic piecewise constant signals. Ann. Statist. 48 (2020), no. 2, 629--654. doi:10.1214/18-AOS1792.

Export citation


  • Amelunxen, D., Lotz, M., McCoy, M. B. and Tropp, J. A. (2014). Living on the edge: Phase transitions in convex programs with random data. Inf. Inference 3 224–294.
  • Arias-Castro, E., Donoho, D. L. and Huo, X. (2005). Near-optimal detection of geometric objects by fast multiscale methods. IEEE Trans. Inform. Theory 51 2402–2425.
  • Bellec, P. C. (2018). Sharp oracle inequalities for least squares estimators in shape restricted regression. Ann. Statist. 46 745–780.
  • Bellec, P. C. and Tsybakov, A. B. (2015). Sharp oracle bounds for monotone and convex regression through aggregation. J. Mach. Learn. Res. 16 1879–1892.
  • Bickel, P. J. and Fan, J. (1996). Some problems on the estimation of unimodal densities. Statist. Sinica 6 23–45.
  • Birgé, L. (1997). Estimation of unimodal densities without smoothness assumptions. Ann. Statist. 25 970–981.
  • Birgé, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113–150.
  • Birgé, L. and Massart, P. (2001). Gaussian model selection. J. Eur. Math. Soc. (JEMS) 3 203–268.
  • Boyarshinov, V. and Magdon-Ismail, M. (2006). Linear time isotonic and unimodal regression in the $L_{1}$ and $L_{\infty}$ norms. J. Discrete Algorithms 4 676–691.
  • Boysen, L., Kempe, A., Liebscher, V., Munk, A. and Wittich, O. (2009). Consistencies and rates of convergence of jump-penalized least squares estimators. Ann. Statist. 37 157–183.
  • Chatterjee, S., Guntuboyina, A. and Sen, B. (2015). On risk bounds in isotonic and other shape restricted regression problems. Ann. Statist. 43 1774–1800.
  • Chatterjee, S. and Lafferty, J. (2019). Adaptive risk bounds in unimodal regression. Bernoulli 25 1–25.
  • Csörgő, M. and Horváth, L. (1997). Limit Theorems in Change-Point Analysis. Wiley Series in Probability and Statistics. Wiley, Chichester.
  • Donoho, D. L. and Johnstone, I. M. (1994). Minimax risk over $l_{p}$-balls for $l_{q}$-error. Probab. Theory Related Fields 99 277–303.
  • Friedrich, F., Kempe, A., Liebscher, V. and Winkler, G. (2008). Complexity penalized $M$-estimation: Fast computation. J. Comput. Graph. Statist. 17 201–224.
  • Gao, C., Han, F. and Zhang, C.-H (2020). Supplement to “On estimation of isotonic piecewise constant signals.”
  • Groeneboom, P. and Jongbloed, G. (2014). Nonparametric Estimation Under Shape Constraints. Cambridge Series in Statistical and Probabilistic Mathematics 38. Cambridge Univ. Press, New York.
  • Groeneboom, P. and Wellner, J. A. (1992). Information Bounds and Nonparametric Maximum Likelihood Estimation. DMV Seminar 19. Birkhäuser, Basel.
  • Haiminen, N., Gionis, A. and Laasonen, K. (2008). Algorithms for unimodal segmentation with applications to unimodality detection. Knowl. Inf. Syst. 14 39–57.
  • Han, Q. and Wellner, J. A. (2016). Multivariate convex regression: Global risk bounds and adaptation. Available at arXiv:1601.06844.
  • Ibragimov, I. A. and Has’ Minskii, R. Z. (2013). Statistical Estimation: Asymptotic Theory. Springer.
  • Jewell, S. and Witten, D. (2018). Exact spike train inference via $\ell_{0}$ optimization. Ann. Appl. Stat. 12 2457–2482.
  • Kim, A. K. H., Guntuboyina, A. and Samworth, R. J. (2018). Adaptation in log-concave density estimation. Ann. Statist. 46 2279–2306.
  • Köllmann, C., Bornkamp, B. and Ickstadt, K. (2014). Unimodal regression using Bernstein–Schoenberg splines and penalties. Biometrics 70 783–793.
  • Lehmann, E. L. and Casella, G. (1998). Theory of Point Estimation, 2nd ed. Springer Texts in Statistics. Springer, New York.
  • Li, H., Munk, A. and Sieling, H. (2016). FDR-control in multiscale change-point segmentation. Electron. J. Stat. 10 918–959.
  • Mair, P., Hornik, K. and de Leeuw, J. (2009). Isotone optimization in R: Pool-adjacent-violators algorithm (PAVA) and active set methods. J. Stat. Softw. 32 1–24.
  • Meyer, M. and Woodroofe, M. (2000). On the degrees of freedom in shape-restricted regression. Ann. Statist. 28 1083–1104.
  • Raskutti, G., Wainwright, M. J. and Yu, B. (2011). Minimax rates of estimation for high-dimensional linear regression over $\ell_{q}$-balls. IEEE Trans. Inform. Theory 57 6976–6994.
  • Rigollet, P. and Tsybakov, A. B. (2012). Sparse estimation by exponential weighting. Statist. Sci. 27 558–575.
  • Robertson, T., Wright, F. T. and Dykstra, R. L. (1988). Order Restricted Statistical Inference. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. Wiley, Chichester.
  • Salanti, G. and Ulm, K. (2003). A nonparametric changepoint model for stratifying continuous variables under order restrictions and binary outcome. Stat. Methods Med. Res. 12 351–367.
  • Schell, M. J. and Singh, B. (1997). The reduced monotonic regression method. J. Amer. Statist. Assoc. 92 128–135.
  • Shoung, J.-M. and Zhang, C.-H. (2001). Least squares estimators of the mode of a unimodal regression function. Ann. Statist. 29 648–665.
  • Silvapulle, M. J. and Sen, P. K. (2011). Constrained Statistical Inference: Order, Inequality, and Shape Constraints. Wiley.
  • Stout, Q. F. (2008). Unimodal regression via prefix isotonic regression. Comput. Statist. Data Anal. 53 289–297.
  • Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation. Springer Series in Statistics. Springer, New York.
  • Zhang, C.-H. (2002). Risk bounds in isotonic regression. Ann. Statist. 30 528–555.

Supplemental materials

  • Supplement to “On estimation of isotonic piecewise constant signals”. This supplement contains proofs of remaining results in Section 4, as well as some auxiliary lemmas.