We establish general sufficient conditions for the $L_2$-consistency of multivariate histogram regression estimates based on data-dependent partitions. These same conditions insure the consistency of partitioning regression estimates based on local polynomial fits, and, with an additional regularity assumption, the consistency of histogram estimates for conditional medians.
Our conditions require shrinking cells, subexponential growth of a combinatorial complexity measure and sublinear growth of restricted cell counts. It is not assumed that the cells of every partition be rectangles with sides parallel to the coordinate axis or that each cell contain a minimum number of points. Response variables are assumed to be bounded throughout.
Our results may be applied to a variety of partitioning schemes. We established the consistency of histograms regression estimates based on cubic partitions with data-dependent offsets, k-thresholding in one dimension and empirically optimal nearest-neighbor clustering schemes. In addition, it is shown that empirically optimal regression trees are consistent when the size of the trees grows with the number of samples at an appropriate rate.
References
ANDERSON, T. W. 1966 . Some nonparametric multivariate procedures based on statistically Z . equivalent blocks. In Multivariate Analy sis P. R. Krishnaiah, ed. 5 27. Academic Press, New York. Z .
BRIEMAN, L., FRIEDMAN, J. H., OLSHEN, R. A. and STONE, C. J. 1984 . Classification and Regression Trees. Wadsworth, Belmont, CA.
CHAUDHURI, P., HUANG, M.-C., LOH, W.-Y. and YAO, R. 1994 . Piecewise poly nomial regression trees. Statist. Sinica 4 143 167. Z .
COVER, T. M. 1965 . Geometrical and statistical properties of sy stems of linear inequalities with applications in pattern recognition. IEEE Transaction on Electronic Computers 14 326 334. Z .
DEVROy E, L. 1988 . Automatic pattern recognition: a study of the probability of error. IEEE Transactions on Pattern Analy sis and Machine Intelligence 10 530 543. Z .
DEVROy E, L. and Gy ORFI, L. 1985 . Distribution-free exponential bounds on the l error of ¨ 1 partitioning estimates of a regression function. In Proceedings of the Fourth PannoZ nian Sy mposium on Mathematical Statistics F. Konecny, J. Mogy orodi, and W. Wertz, ´ . eds. 67 76. Akademiai Kiado, Budapest. ´ ´ Z .
GERSHO, A. and GRAY, R. M. 1992 . Vector Quantization and Signal Compression. Kluwer, Dordrecht. Z .
GESSAMAN, M. P. 1970 . A consistent nonparametric multivariate density estimator based on statistically equivalent blocks. Ann. Math. Statist. 41 1344 1346. Z .
GORDON, L. and OLSHEN, R. 1978 . Asy mptotically efficient solutions to the classification problem. Ann. Statist. 6 515 533. Z .
GORDON, L. and OLSHEN, R. 1980 . Consistent nonparametric regression from recursive partitioning schemes. J. Multivariate Anal. 10 611 627. Z .
GORDON, L. and OLSHEN, R. 1984 . Almost sure consistent nonparametric regression from recursive partitioning schemes. J. Multivariate Anal. 15 147 163. Z .
HARTIGAN, J. A. 1975 . Clustering Algorithms. Wiley, New York. Z .
LINDE, Y., BUZO, A. and GRAY, R. M. 1980 . An algorithm for vector quantizer design. IEEE Trans. Comm. 28 84 95. Z .
LUGOSI, G. and NOBEL, A. B. 1995 . Consistency of data-driven histogram methods for density estimation and classification. Ann. Statist. To appear. Z .
PARTHASARATHY, K. R. and BHATTACHARy A, P. K. 1961 . Some limit theorems in regression theory. Sankhy a Ser. A 23 91 102. Z .
PATRICK, E. A. and FISHER, F. P. 1967 . Introduction to the performance of distribution-free conditional risk learning sy stems. Technical Report TR-EE-67-12, Purdue Univ. Z .
POLLARD, D. 1984 . Convergence of Stochastic Processes. Springer, New York. Z . Z .
STONE, C. J. 1977 . Consistent nonparametric regression with discussion . Ann. Statist. 5 595 645. Z .
STONE, C. J. 1985 . An asy mptotically optimal histogram selection rule. In Proceedings of the Z Berkeley Conference in Honor of Jerzy Ney man and Jack Kiefer L. Le Cam and R. A. . Olshen, eds. 513 520. Wadsworth, Belmont, CA. Z .
VAPNIK, V. N. and CHERVONENKIS, A. YA. 1971 . On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16 264 280. Z .
VAPNIK, V. N. and CHERVONENKIS, A. YA. 1981 . Necessary and sufficient conditions for the uniform convergence of means to their expectations. Theory Probab. Appl. 26 532 553. Z .
ZHAO, L. C., KRISHNAIAH, P. R. and CHEN, X. R. 1990 . Almost sure L -norm convergence for r data-based histogram density estimates. Theory Probab. Appl. 35 396 403.
CHAPEL HILL, NORTH CAROLINA 27599-3260 E-MAIL: nobel@stat.unc.edu
