Translator Disclaimer
October 2021 Adaptive estimation of multivariate piecewise polynomials and bounded variation functions by optimal decision trees
Sabyasachi Chatterjee, Subhajit Goswami
Author Affiliations +
Ann. Statist. 49(5): 2531-2551 (October 2021). DOI: 10.1214/20-AOS2045

Abstract

Proposed by Donoho (Ann. Statist. 25 (1997) 1870–1911), Dyadic CART is a nonparametric regression method which computes a globally optimal dyadic decision tree and fits piecewise constant functions in two dimensions. In this article, we define and study Dyadic CART and a closely related estimator, namely Optimal Regression Tree (ORT), in the context of estimating piecewise smooth functions in general dimensions in the fixed design setup. More precisely, these optimal decision tree estimators fit piecewise polynomials of any given degree. Like Dyadic CART in two dimensions, we reason that these estimators can also be computed in polynomial time in the sample size N via dynamic programming. We prove oracle inequalities for the finite sample risk of Dyadic CART and ORT, which imply tight risk bounds for several function classes of interest. First, they imply that the finite sample risk of ORT of order r0 is always bounded by CklogNN whenever the regression function is piecewise polynomial of degree r on some reasonably regular axis aligned rectangular partition of the domain with at most k rectangles. Beyond the univariate case, such guarantees are scarcely available in the literature for computationally efficient estimators. Second, our oracle inequalities uncover minimax rate optimality and adaptivity of the Dyadic CART estimator for function spaces with bounded variation. We consider two function spaces of recent interest where multivariate total variation denoising and univariate trend filtering are the state of the art methods. We show that Dyadic CART enjoys certain advantages over these estimators while still maintaining all their known guarantees.

Funding Statement

The first author was supported by NSF Grant DMS-1916375. The second author was supported by an IDEX grant from Paris–Saclay and partially by a grant from the Infosys Foundation.

Acknowledgments

Author names are sorted alphabetically. S.G.’s research was carried out in part as a member of the Infosys-Chandrasekharan virtual center for Random Geometry, supported by a grant from the Infosys Foundation. We thank the anonymous referees for their numerous helpful remarks and suggestions on an earlier manuscript of the paper. We also thank Adityanand Guntuboyina for many helpful comments. The project started when SG was a postdoctoral fellow at the Institut des Hautes Études Scientifiques (IHES).

Citation

Download Citation

Sabyasachi Chatterjee. Subhajit Goswami. "Adaptive estimation of multivariate piecewise polynomials and bounded variation functions by optimal decision trees." Ann. Statist. 49 (5) 2531 - 2551, October 2021. https://doi.org/10.1214/20-AOS2045

Information

Received: 1 December 2019; Revised: 1 December 2020; Published: October 2021
First available in Project Euclid: 12 November 2021

Digital Object Identifier: 10.1214/20-AOS2045

Subjects:
Primary: 62G05

Keywords: bounded variation function estimation , Dyadic CART , Optimal decision trees , oracle risk bounds , piecewise polynomial fitting

Rights: Copyright © 2021 Institute of Mathematical Statistics

JOURNAL ARTICLE
21 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

SHARE
Vol.49 • No. 5 • October 2021
Back to Top