Electronic Journal of Statistics

On convex least squares estimation when the truth is linear

Yining Chen and Jon A. Wellner

Full-text: Open access


We prove that the convex least squares estimator (LSE) attains a $n^{-1/2}$ pointwise rate of convergence in any region where the truth is linear. In addition, the asymptotic distribution can be characterized by a modified invelope process. Analogous results hold when one uses the derivative of the convex LSE to perform derivative estimation. These asymptotic results facilitate a new consistent testing procedure on the linearity against a convex alternative. Moreover, we show that the convex LSE adapts to the optimal rate at the boundary points of the region where the truth is linear, up to a log-log factor. These conclusions are valid in the context of both density estimation and regression function estimation.

Article information

Electron. J. Statist., Volume 10, Number 1 (2016), 171-209.

Received: December 2014
First available in Project Euclid: 17 February 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62E20: Asymptotic distribution theory 62G07: Density estimation 62G08: Nonparametric regression 62G10: Hypothesis testing 62G20: Asymptotic properties
Secondary: 60G15: Gaussian processes

Adaptive estimation convexity density estimation least squares regression function estimation shape constraint


Chen, Yining; Wellner, Jon A. On convex least squares estimation when the truth is linear. Electron. J. Statist. 10 (2016), no. 1, 171--209. doi:10.1214/15-EJS1098. https://projecteuclid.org/euclid.ejs/1455715960

Export citation


  • Balabdaoui, F. (2007) Consistent estimation of a convex density at the origin., Math. Methods Statist., 16, 77–95.
  • Balabdaoui, F. (2014) Global convergence of the log-concave MLE when the truth is geometric., J. Nonparametric. Statist., 26, 21–59.
  • Balabdaoui, F., Jankowski, H., Rufibach, K. and Pavlides, M. (2013). Asymptotic distribution of the discrete log-concave MLE and related applications., J. Roy. Statist. Soc. Ser. B, 75, 769–790.
  • Balabdaoui, F. and Rufibach, K. (2008) A second Marshall inequality in convex estimation., Statist. Probab. Lett., 78, 118–126.
  • Balabdaoui, F., Rufibach, K. and Wellner, J. A. (2009) Limit distribution theory for maximum likelihood estimation of a log-concave density., Ann. Statist., 37, 1299–1331.
  • Balabdaoui, F. and Wellner, J. A. (2007) Estimation of a $k$-monotone density: limit distribution theory and the spline connection., Ann. Statist., 35, 2536–2564.
  • Balabdaoui, F. and Wellner, J. A. (2014) Chernoff’s density is log-concave., Bernoulli, 20, 231–244.
  • Cai, T. T. and Low, M. (2015) A framework for estimation of convex functions., Statist. Sinica., 25, 423–456.
  • Cator, E. (2011) Adaptivity and optimality of the monotone least-squares estimator., Bernoulli, 17, 714-735.
  • Carolan, C. and Dykstra, R. (1999) Asymptotic behavior of the Grenander estimator at density flat regions., Canad. J. Statist., 27, 557–566.
  • Chen, Y. and Samworth, R. J. (2013) Smoothed log-concave maximum likelihood estimation with applications., Statist. Sinica., 23, 1373–1398.
  • Chen, Y. and Samworth, R. J. (2016) Generalized additive and index models with shape constraints., J. R. Stat. Soc. Ser. B., to appear. Available at http://arxiv.org/abs/1404.2957.
  • Csörgő, M. and Horvath, L. (1993) Weighted approximations in probability and statistics. John Wiley & Sons Inc., Chichester, England.
  • Cule, M. and Samworth, R. (2010) Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density., Electron. J. Statist., 4, 254–270.
  • Doss, C. and Wellner, J. A. (2016) Global rates of convergence of the MLEs of log-concave and s-concave densities., Ann. Statist., to appear. Available at http://arxiv.org/abs/1306.1438.
  • Dudley, R. M. (1999) Uniform central limit theorems. Cambridge University Press, Cambridge, U.K.
  • Dümbgen, L., Rufibach, K. and Wellner, J. A. (2007) Marshall’s lemma for convex density estimation. In, Asymptotics: particles, processes and inverse problems: festschrift for Piet Groeneboom (E. Cator, G. Jongbloed, C. Kraaikamp, R. Lopuhaä and J.A. Wellner, eds.), 101–107. Institute of Mathematical Statistics, Ohio.
  • Dümbgen, L., Samworth, R. and Schuhmacher, D. (2011) Approximation by log-concave distributions with applications to regression., Ann. Statist., 39, 702-730.
  • Gao, F. and Wellner, J. A. (2009) On the rate of convergence of the maximum likelihood estimator of a $k$-monotone density., Science in China Series A: Mathematics, 52, 1–14.
  • Groeneboom, P. (1985) Estimating a monotone density. In, Proceedings of the Berkeley conference in honor of Jerzy Neyman and Jack Kiefer, 539–555, Wadsworth, Belmont, CA.
  • Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2001a) A canonical process for estimation of convex functions: the “invelope” of integrated Brownian motion $+t^4$., Ann. Statist., 29, 1620–1652.
  • Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2001b) Estimation of a convex function: characterizations and asymptotic theory., Ann. Statist., 29, 1653–1698.
  • Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2008) The support reduction algorithm for computing nonparametric function estimates in mixture models., Scand. J. Statist., 35, 385-399.
  • Groeneboom, P. and Pyke, R. (1983) Asymptotic normality of statistics based on the convex minorants of empirical distribution functions., Ann. Prob., 11, 328–345.
  • Guntuboyina, A. and Sen, B. (2013) Covering numbers for convex functions., IEEE Trans. Inf. Theory, 59, 1957–1965.
  • Guntuboyina, A. and Sen, B. (2015) Global risk bounds and adaptation in univariate convex regression., Prob. Theory Related Fields, 163, 379–411.
  • Hanson, D.L. and Pledger, G. (1976) Consistency in concave regression., Ann. Statist., 4, 1038–1050.
  • Hildreth, C. (1954) Point estimators of ordinates of concave functions., J. Amer. Statist. Assoc., 49, 598–619.
  • Lachal, A. (1997) Local asymptotic classes for the successive primitives of Brownian motion, Ann. Prob., 25, 1712–1734.
  • Jankowski, H. (2014) Convergence of linear functionals of the Grenander estimator under misspecification., Ann. Statist., 42, 625–653.
  • Kim, J. and Pollard, D. (1990) Cube root asymptotics., Ann. Statist., 18, 191–219.
  • Kim, A. K. H. and Samworth, R. J. (2014) Global rates of convergence in log-concave density estimation. Available at, http://arxiv.org/abs/1404.2298.
  • Mammen, E. (1991) Nonparametric regression under qualitative smoothness assumptions., Ann. Statist., 19, 741–759.
  • Meyer, M. C. (2003). A test for linear versus convex regression function using shape-restricted regression., Biometrika, 90, 223–232.
  • Meyer, M. C. (2013). Semi-parametric additive constrained regression., J. Nonparametric Statist., 25, 715–743.
  • Rockafellar, R. T. (1997), Convex analysis. Princeton University Press, Princeton, NJ.
  • Shorack, G. R. and Wellner, J. A. (1986), Empirical processes with applications to statistics. John Wiley & Sons Inc, New York, NY.
  • Sen, B. and Meyer, M. C. (2013). Testing against a parametric regression function using ideas from shape restricted estimation. Available at, http://arxiv.org/abs/1311.6849.