## Bernoulli

### Adaptive risk bounds in unimodal regression

#### Abstract

We study the statistical properties of the least squares estimator in unimodal sequence estimation. Although closely related to isotonic regression, unimodal regression has not been as extensively studied. We show that the unimodal least squares estimator is adaptive in the sense that the risk scales as a function of the number of values in the true underlying sequence. Such adaptivity properties have been shown for isotonic regression by Chatterjee et al. (Ann. Statist. 43 (2015) 1774–1800) and Bellec (Sharp oracle inequalities for Least Squares estimators in shape restricted regression (2016)). A technical complication in unimodal regression is the non-convexity of the underlying parameter space. We develop a general variational representation of the risk that holds whenever the parameter space can be expressed as a finite union of convex sets, using techniques that may be of interest in other settings.

#### Article information

Source
Bernoulli, Volume 25, Number 1 (2019), 1-25.

Dates
Received: July 2016
Revised: December 2016
First available in Project Euclid: 12 December 2018

Permanent link to this document
https://projecteuclid.org/euclid.bj/1544605234

Digital Object Identifier
doi:10.3150/16-BEJ922

Mathematical Reviews number (MathSciNet)
MR3892309

Zentralblatt MATH identifier
07007197

#### Citation

Chatterjee, Sabyasachi; Lafferty, John. Adaptive risk bounds in unimodal regression. Bernoulli 25 (2019), no. 1, 1--25. doi:10.3150/16-BEJ922. https://projecteuclid.org/euclid.bj/1544605234

#### References

• [1] Amelunxen, D., Lotz, M., McCoy, M.B. and Tropp, J.A. (2014). Living on the edge: Phase transitions in convex programs with random data. Inf. Inference 3 224–294.
• [2] Balabdaoui, F. and Jankowski, H. (2015). Maximum likelihood estimation of a unimodal probability mass function.
• [3] Barlow, R.E., Bartholomew, D.J., Bremner, J.M. and Brunk, H.D. (1972). Statistical Inference Under Order Restrictions. The Theory and Application of Isotonic Regression. Wiley Series in Probability and Mathematical Statistics. New York: Wiley.
• [4] Bellec, P.C. (2015). Sharp oracle inequalities for least squares estimators in shape restricted regression. Available at arXiv:1510.08029.
• [5] Bellec, P.C. (2016). Sharp oracle inequalities for Least Squares estimators in shape restricted regression. Available at arXiv:1510.08029v2.
• [6] Bickel, P.J. and Fan, J. (1996). Some problems on the estimation of unimodal densities. Statist. Sinica 6 23–45.
• [7] Birgé, L. (1997). Estimation of unimodal densities without smoothness assumptions. Ann. Statist. 25 970–981.
• [8] Birgé, L. and Massart, P. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113–150.
• [9] Boucheron, S., Lugosi, G. and Massart, P. (2013). Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford: Oxford Univ. Press. With a foreword by Michel Ledoux.
• [10] Boyarshinov, V. and Magdon-Ismail, M. (2006). Linear time isotonic and unimodal regression in the $L_{1}$ and $L_{\infty}$ norms. J. Discrete Algorithms 4 676–691.
• [11] Bro, R. and Sidiropoulos, N. (1998). Least squares algorithms under unimodality and non-negativity constraints. J. Chemom. 12 223–247.
• [12] Chatterjee, S. (2014). A new perspective on least squares under convex constraint. Ann. Statist. 42 2340–2381.
• [13] Chatterjee, S., Guntuboyina, A. and Sen, B. (2015). On matrix estimation under monotonicity constraints. Available at arXiv:1506.03430.
• [14] Chatterjee, S., Guntuboyina, A. and Sen, B. (2015). On risk bounds in isotonic and other shape restricted regression problems. Ann. Statist. 43 1774–1800.
• [15] Chatterjee, S. and Mukherjee, S. (2016). On estimation in tournaments and graphs under monotonicity constraints. Available at arXiv:1603.04556.
• [16] Donoho, D. (1991). Gelfand $n$-widths and the method of least squares Technical report, Dept. Statistics, Univ. California, Berkeley.
• [17] Eggermont, P.P.B. and LaRiccia, V.N. (2000). Maximum likelihood estimation of smooth monotone and unimodal densities. Ann. Statist. 28 922–947.
• [18] Flammarion, N., Mao, C. and Rigollet, P. (2016). Optimal rates of statistical seriation. Available at arXiv:1607.02435v1.
• [19] Frisén, M. (1986). Unimodal regression. J. R. Stat. Soc. Ser. D Stat. 479–485.
• [20] Gao, F. and Wellner, J.A. (2007). Entropy estimate for high-dimensional monotonic functions. J. Multivariate Anal. 98 1751–1764.
• [21] Grotzinger, S.J. and Witzgall, C. (1984). Projections onto order simplexes. Appl. Math. Optim. 12 247–270.
• [22] Guntuboyina, A. and Sen, B. (2015). Global risk bounds and adaptation in univariate convex regression. Probab. Theory Related Fields 163 379–411.
• [23] Köllmann, C., Bornkamp, B. and Ickstadt, K. (2014). Unimodal regression using Bernstein–Schoenberg splines and penalties. Biometrics 70 783–793.
• [24] Laurent, B. and Massart, P. (2000). Adaptive estimation of a quadratic functional by model selection. Ann. Statist. 28 1302–1338.
• [25] Ledoux, M. (2001). The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89. Providence, RI: Amer. Math. Soc.
• [26] Meyer, M. and Woodroofe, M. (2000). On the degrees of freedom in shape-restricted regression. Ann. Statist. 28 1083–1104.
• [27] Meyer, M.C. (2001). An alternative unimodal density estimator with a consistent estimate of the mode. Statist. Sinica 11 1159–1174.
• [28] Shah, N.B., Balakrishnan, S., Guntuboyina, A. and Wainwright, M.J. (2017). Stochastically transitive models for pairwise comparisons: Statistical and computational issues. IEEE Trans. Inform. Theory 63 934–959.
• [29] Shoung, J.-M. and Zhang, C.-H. (2001). Least squares estimators of the mode of a unimodal regression function. Ann. Statist. 29 648–665.
• [30] Stout, Q.F. (2000). Optimal algorithms for unimodal regression. Ann Arbor 1001 48109–2122.
• [31] Stout, Q.F. (2008). Unimodal regression via prefix isotonic regression. Comput. Statist. Data Anal. 53 289–297.
• [32] van de Geer, S. (1990). Estimating a regression function. Ann. Statist. 18 907–924.
• [33] van de Geer, S. (1993). Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Statist. 21 14–44.
• [34] van de Geer, S.A. (2000). Applications of Empirical Process Theory. Cambridge Series in Statistical and Probabilistic Mathematics 6. Cambridge: Cambridge Univ. Press.
• [35] Wang, Y. (1996). The $L_{2}$ risk of an isotonic estimate. Comm. Statist. Theory Methods 25 281–294.
• [36] Zhang, C.-H. (2002). Risk bounds in isotonic regression. Ann. Statist. 30 528–555.