The Annals of Statistics

Histogram regression estimation using data-dependent partitions

Andrew Nobel

Full-text: Open access

Abstract

We establish general sufficient conditions for the $L_2$-consistency of multivariate histogram regression estimates based on data-dependent partitions. These same conditions insure the consistency of partitioning regression estimates based on local polynomial fits, and, with an additional regularity assumption, the consistency of histogram estimates for conditional medians.

Our conditions require shrinking cells, subexponential growth of a combinatorial complexity measure and sublinear growth of restricted cell counts. It is not assumed that the cells of every partition be rectangles with sides parallel to the coordinate axis or that each cell contain a minimum number of points. Response variables are assumed to be bounded throughout.

Our results may be applied to a variety of partitioning schemes. We established the consistency of histograms regression estimates based on cubic partitions with data-dependent offsets, k-thresholding in one dimension and empirically optimal nearest-neighbor clustering schemes. In addition, it is shown that empirically optimal regression trees are consistent when the size of the trees grows with the number of samples at an appropriate rate.

Article information

Source
Ann. Statist., Volume 24, Number 3 (1996), 1084-1105.

Dates
First available in Project Euclid: 20 September 2002

Permanent link to this document
https://projecteuclid.org/euclid.aos/1032526958

Digital Object Identifier
doi:10.1214/aos/1032526958

Mathematical Reviews number (MathSciNet)
MR1401839

Zentralblatt MATH identifier
0862.62038

Subjects
Primary: 62G07: Density estimation
Secondary: 62H30: Classification and discrimination; cluster analysis [See also 68T10, 91C20]

Keywords
Regression estimation histogram estimates regression trees clustering Vapnik-Chervonenkis theory data-dependent partitions

Citation

Nobel, Andrew. Histogram regression estimation using data-dependent partitions. Ann. Statist. 24 (1996), no. 3, 1084--1105. doi:10.1214/aos/1032526958. https://projecteuclid.org/euclid.aos/1032526958


Export citation

References

  • ANDERSON, T. W. 1966. Some nonparametric multivariate procedures based on statistically Z. equivalent blocks. In Multivariate Analy sis P. R. Krishnaiah, ed. 5 27. Academic Press, New York. Z.
  • BRIEMAN, L., FRIEDMAN, J. H., OLSHEN, R. A. and STONE, C. J. 1984. Classification and Regression Trees. Wadsworth, Belmont, CA.
  • CHAUDHURI, P., HUANG, M.-C., LOH, W.-Y. and YAO, R. 1994. Piecewise poly nomial regression trees. Statist. Sinica 4 143 167. Z.
  • COVER, T. M. 1965. Geometrical and statistical properties of sy stems of linear inequalities with applications in pattern recognition. IEEE Transaction on Electronic Computers 14 326 334. Z.
  • DEVROy E, L. 1988. Automatic pattern recognition: a study of the probability of error. IEEE Transactions on Pattern Analy sis and Machine Intelligence 10 530 543. Z.
  • DEVROy E, L. and Gy ORFI, L. 1985. Distribution-free exponential bounds on the l error of ¨ 1 partitioning estimates of a regression function. In Proceedings of the Fourth PannoZ nian Sy mposium on Mathematical Statistics F. Konecny, J. Mogy orodi, and W. Wertz, ´. eds. 67 76. Akademiai Kiado, Budapest. ´ ´ Z.
  • GERSHO, A. and GRAY, R. M. 1992. Vector Quantization and Signal Compression. Kluwer, Dordrecht. Z.
  • GESSAMAN, M. P. 1970. A consistent nonparametric multivariate density estimator based on statistically equivalent blocks. Ann. Math. Statist. 41 1344 1346. Z.
  • GORDON, L. and OLSHEN, R. 1978. Asy mptotically efficient solutions to the classification problem. Ann. Statist. 6 515 533. Z.
  • GORDON, L. and OLSHEN, R. 1980. Consistent nonparametric regression from recursive partitioning schemes. J. Multivariate Anal. 10 611 627. Z.
  • GORDON, L. and OLSHEN, R. 1984. Almost sure consistent nonparametric regression from recursive partitioning schemes. J. Multivariate Anal. 15 147 163. Z.
  • HARTIGAN, J. A. 1975. Clustering Algorithms. Wiley, New York. Z.
  • LINDE, Y., BUZO, A. and GRAY, R. M. 1980. An algorithm for vector quantizer design. IEEE Trans. Comm. 28 84 95. Z.
  • LUGOSI, G. and NOBEL, A. B. 1995. Consistency of data-driven histogram methods for density estimation and classification. Ann. Statist. To appear. Z.
  • PARTHASARATHY, K. R. and BHATTACHARy A, P. K. 1961. Some limit theorems in regression theory. Sankhy a Ser. A 23 91 102. Z.
  • PATRICK, E. A. and FISHER, F. P. 1967. Introduction to the performance of distribution-free conditional risk learning sy stems. Technical Report TR-EE-67-12, Purdue Univ. Z.
  • POLLARD, D. 1984. Convergence of Stochastic Processes. Springer, New York. Z. Z.
  • STONE, C. J. 1977. Consistent nonparametric regression with discussion. Ann. Statist. 5 595 645. Z.
  • STONE, C. J. 1985. An asy mptotically optimal histogram selection rule. In Proceedings of the Z Berkeley Conference in Honor of Jerzy Ney man and Jack Kiefer L. Le Cam and R. A.. Olshen, eds. 513 520. Wadsworth, Belmont, CA. Z.
  • VAPNIK, V. N. and CHERVONENKIS, A. YA. 1971. On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16 264 280. Z.
  • VAPNIK, V. N. and CHERVONENKIS, A. YA. 1981. Necessary and sufficient conditions for the uniform convergence of means to their expectations. Theory Probab. Appl. 26 532 553. Z.
  • ZHAO, L. C., KRISHNAIAH, P. R. and CHEN, X. R. 1990. Almost sure L -norm convergence for r data-based histogram density estimates. Theory Probab. Appl. 35 396 403.
  • CHAPEL HILL, NORTH CAROLINA 27599-3260 E-MAIL: nobel@stat.unc.edu