The Annals of Statistics

Nonparametric least squares estimation of a multivariate convex regression function

Emilio Seijo and Bodhisattva Sen

Full-text: Open access


This paper deals with the consistency of the nonparametric least squares estimator of a convex regression function when the predictor is multidimensional. We characterize and discuss the computation of such an estimator via the solution of certain quadratic and linear programs. Mild sufficient conditions for the consistency of this estimator and its subdifferentials in fixed and stochastic design regression settings are provided.

Article information

Ann. Statist., Volume 39, Number 3 (2011), 1633-1657.

First available in Project Euclid: 25 July 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G08: Nonparametric regression 62G05: Estimation
Secondary: 62G20: Asymptotic properties

Consistency linear program semidefinite quadratic program shape restricted estimation subdifferentials


Seijo, Emilio; Sen, Bodhisattva. Nonparametric least squares estimation of a multivariate convex regression function. Ann. Statist. 39 (2011), no. 3, 1633--1657. doi:10.1214/10-AOS852.

Export citation


  • Allon, G., Beenstock, M., Hackman, S., Passy, U. and Shapiro, A. (2007). Nonparametric estimation of concave production technologies by entropic methods. J. Appl. Econometrics 22 795–816.
  • Banker, R. D. and Maindiratta, A. (1992). Maximum likelihood estimation of monotone and concave production frontiers. J. Productiv. Anal. 3 401–415.
  • Beresteanu, A. (2007). Nonparametric estimation of regression functions under restrictions on partial derivatives. Available at
  • Birke, M. and Dette, H. (2007). Estimating a convex function in nonparametric regression. Scand. J. Stat. 34 384–404.
  • Bronšteĭn, E. M. (1978). Extremal convex functions. Sibirsk. Mat. Zh. 19 10–18.
  • Brunk, H. D. (1955). Maximum likelihood estimates of monotone parameters. Ann. Math. Statist. 26 607–616.
  • Brunk, H. D. (1970). Estimation of isotonic regression. In Nonparametric Techniques in Statistical Inference 177–197. Cambridge Univ. Press, New York.
  • Chung, K. L. (2001). A Course in Probability Theory. Academic Press, San Diego, CA.
  • Conway, J. (1985). A Course in Functional Analysis. Springer, New York.
  • Cule, M. and Samworth, R. (2010). Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density. Electron. J. Stat. 4 254–270.
  • Cule, M., Samworth, R. and Stewart, M. (2010). Maximum likelihood estimation of a multidimensional log-concave density. J. R. Stat. Soc. Ser. B Stat. Methodol. 72 545–607.
  • Dudley, R. M. (1977). On second derivatives of convex functions. Math. Scand. 41 159–174.
  • Folland, G. (1999). Real Analysis: Modern Techniques and Their Applications. Wiley, New York.
  • Grenander, U. (1956). On the theory of mortality measurement. II. Skand. Aktuarietidskr. 39 125–153.
  • Groeneboom, P., Jongbloed, G. and Wellner, J. (2001). Estimation of a convex function: Characterizations and asymptotic theory. Ann. Statist. 29 1653–1698.
  • Hanson, D. L. and Pledger, G. (1976). Consistency in concave regression. Ann. Statist. 4 1038–1050.
  • Hildreth, C. (1954). Point estimates of ordinates of concave functions. J. Amer. Statist. Assoc. 49 598–619.
  • Johansen, S. (1974). The extremal convex functions. Math. Scand. 41 61–68.
  • Kuosmanen, T. (2008). Representation theorem for convex nonparametric least squares. Econom. J. 11 308–325.
  • Luenberger, D. (1984). Linear and Nonlinear Programming. Addison-Wesley, Reading, MA.
  • Mammen, E. (1991). Nonparametric regression under qualitative smoothness assumptions. Ann. Statist. 19 741–759.
  • Matzkin, R. L. (1991). Semiparametric estimation of monotone concave utility functions for polychotomous choice models. Econometrica 59 1351–1327.
  • Matzkin, R. L. (1993). Nonparametric identification and estimation of polychotomous choice models. J. Econometrics 58 137–168.
  • Nocedal, J. and Wright, S. (1999). Numerical Optimization. Springer, New York.
  • Rockafellar, T. R. (1970). Convex Analysis. Princeton Univ. Press, Princeton, NJ.
  • Sarath, B. and Maindiratta, A. (1997). On the consistency of maximum likelihood estimation of monotone and concave production frontiers. J. Productiv. Anal. 8 239–246.
  • Schuhmacher, D. and Dümbgen, L. (2010). Consistency of multivariate log-concave density estimators. Statist. Probab. Lett. 80 376–380.
  • Schuhmacher, D., Hüsler, A. and Dümbgen, L. (2009). Multivariate log-concave distributions as a nearly parametric model. Technical report, Univ. Bern. Available at
  • Seijo, E. and Sen, B. (2011). Supplement to “Nonparametric least squares estimation of a multivariate convex regression function.” DOI:10.1214/10-AOS852SUPP.
  • Seregin, A. and Wellner, J. (2010). Nonparametric estimation of multivariate convex-transformed densities. Ann. Statist. 38 3751–3781.
  • Van der Vaart, A. and Wellner, J. (1996). Weak Convergence and Empirical Processes. Springer, New York.
  • Varian, H. (1982). The nonparametric approach to demand analysis. Econometrica 50 945–973.
  • Varian, H. (1984). The nonparametric approach to production analysis. Econometrica 52 579–597.
  • Williams, D. (1991). Probability with Martingales. Cambridge Univ. Press, Cambridge.
  • Zhang, C. H. (2002). Risk bounds in isotonic regression. Ann. Statist. 30 528–555.

Supplemental materials

  • Supplementary material: Supplement to “Nonparametric least squares estimation of a multivariate convex regression function”. The supplementary file contains the proofs of some technical results that were omitted from the main draft due to their length.