The Annals of Statistics

Nonparametric least squares estimation of a multivariate convex regression function

Emilio Seijo and Bodhisattva Sen

Full-text: Open access

Abstract

This paper deals with the consistency of the nonparametric least squares estimator of a convex regression function when the predictor is multidimensional. We characterize and discuss the computation of such an estimator via the solution of certain quadratic and linear programs. Mild sufficient conditions for the consistency of this estimator and its subdifferentials in fixed and stochastic design regression settings are provided.

Article information

Source
Ann. Statist., Volume 39, Number 3 (2011), 1633-1657.

Dates
First available in Project Euclid: 25 July 2011

Permanent link to this document
https://projecteuclid.org/euclid.aos/1311600278

Digital Object Identifier
doi:10.1214/10-AOS852

Mathematical Reviews number (MathSciNet)
MR2850215

Zentralblatt MATH identifier
0132.38905

Subjects
Primary: 62G08: Nonparametric regression 62G05: Estimation
Secondary: 62G20: Asymptotic properties

Keywords
Consistency linear program semidefinite quadratic program shape restricted estimation subdifferentials

Citation

Seijo, Emilio; Sen, Bodhisattva. Nonparametric least squares estimation of a multivariate convex regression function. Ann. Statist. 39 (2011), no. 3, 1633--1657. doi:10.1214/10-AOS852. https://projecteuclid.org/euclid.aos/1311600278


Export citation

References

  • Allon, G., Beenstock, M., Hackman, S., Passy, U. and Shapiro, A. (2007). Nonparametric estimation of concave production technologies by entropic methods. J. Appl. Econometrics 22 795–816.
  • Banker, R. D. and Maindiratta, A. (1992). Maximum likelihood estimation of monotone and concave production frontiers. J. Productiv. Anal. 3 401–415.
  • Beresteanu, A. (2007). Nonparametric estimation of regression functions under restrictions on partial derivatives. Available at http://www.pitt.edu/~arie/shape.pdf.
  • Birke, M. and Dette, H. (2007). Estimating a convex function in nonparametric regression. Scand. J. Stat. 34 384–404.
  • Bronšteĭn, E. M. (1978). Extremal convex functions. Sibirsk. Mat. Zh. 19 10–18.
  • Brunk, H. D. (1955). Maximum likelihood estimates of monotone parameters. Ann. Math. Statist. 26 607–616.
  • Brunk, H. D. (1970). Estimation of isotonic regression. In Nonparametric Techniques in Statistical Inference 177–197. Cambridge Univ. Press, New York.
  • Chung, K. L. (2001). A Course in Probability Theory. Academic Press, San Diego, CA.
  • Conway, J. (1985). A Course in Functional Analysis. Springer, New York.
  • Cule, M. and Samworth, R. (2010). Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density. Electron. J. Stat. 4 254–270.
  • Cule, M., Samworth, R. and Stewart, M. (2010). Maximum likelihood estimation of a multidimensional log-concave density. J. R. Stat. Soc. Ser. B Stat. Methodol. 72 545–607.
  • Dudley, R. M. (1977). On second derivatives of convex functions. Math. Scand. 41 159–174.
  • Folland, G. (1999). Real Analysis: Modern Techniques and Their Applications. Wiley, New York.
  • Grenander, U. (1956). On the theory of mortality measurement. II. Skand. Aktuarietidskr. 39 125–153.
  • Groeneboom, P., Jongbloed, G. and Wellner, J. (2001). Estimation of a convex function: Characterizations and asymptotic theory. Ann. Statist. 29 1653–1698.
  • Hanson, D. L. and Pledger, G. (1976). Consistency in concave regression. Ann. Statist. 4 1038–1050.
  • Hildreth, C. (1954). Point estimates of ordinates of concave functions. J. Amer. Statist. Assoc. 49 598–619.
  • Johansen, S. (1974). The extremal convex functions. Math. Scand. 41 61–68.
  • Kuosmanen, T. (2008). Representation theorem for convex nonparametric least squares. Econom. J. 11 308–325.
  • Luenberger, D. (1984). Linear and Nonlinear Programming. Addison-Wesley, Reading, MA.
  • Mammen, E. (1991). Nonparametric regression under qualitative smoothness assumptions. Ann. Statist. 19 741–759.
  • Matzkin, R. L. (1991). Semiparametric estimation of monotone concave utility functions for polychotomous choice models. Econometrica 59 1351–1327.
  • Matzkin, R. L. (1993). Nonparametric identification and estimation of polychotomous choice models. J. Econometrics 58 137–168.
  • Nocedal, J. and Wright, S. (1999). Numerical Optimization. Springer, New York.
  • Rockafellar, T. R. (1970). Convex Analysis. Princeton Univ. Press, Princeton, NJ.
  • Sarath, B. and Maindiratta, A. (1997). On the consistency of maximum likelihood estimation of monotone and concave production frontiers. J. Productiv. Anal. 8 239–246.
  • Schuhmacher, D. and Dümbgen, L. (2010). Consistency of multivariate log-concave density estimators. Statist. Probab. Lett. 80 376–380.
  • Schuhmacher, D., Hüsler, A. and Dümbgen, L. (2009). Multivariate log-concave distributions as a nearly parametric model. Technical report, Univ. Bern. Available at http://arxiv.org/abs/0907.0250.
  • Seijo, E. and Sen, B. (2011). Supplement to “Nonparametric least squares estimation of a multivariate convex regression function.” DOI:10.1214/10-AOS852SUPP.
  • Seregin, A. and Wellner, J. (2010). Nonparametric estimation of multivariate convex-transformed densities. Ann. Statist. 38 3751–3781.
  • Van der Vaart, A. and Wellner, J. (1996). Weak Convergence and Empirical Processes. Springer, New York.
  • Varian, H. (1982). The nonparametric approach to demand analysis. Econometrica 50 945–973.
  • Varian, H. (1984). The nonparametric approach to production analysis. Econometrica 52 579–597.
  • Williams, D. (1991). Probability with Martingales. Cambridge Univ. Press, Cambridge.
  • Zhang, C. H. (2002). Risk bounds in isotonic regression. Ann. Statist. 30 528–555.

Supplemental materials

  • Supplementary material: Supplement to “Nonparametric least squares estimation of a multivariate convex regression function”. The supplementary file contains the proofs of some technical results that were omitted from the main draft due to their length.