Conditional regression for single-index models

Abstract

The single-index model is a statistical model for intrinsic regression where responses are assumed to depend on a single yet unknown linear combination of the predictors, allowing to express the regression function as $\mathbb{E}[Y|X]=f(⟨v,X⟩)$ for some unknown index vector v and link function f. Conditional methods provide a simple and effective approach to estimate v by averaging moments of X conditioned on Y, but depend on parameters whose optimal choice is unknown and do not provide generalization bounds on f. In this paper we propose a new conditional method converging at $\sqrt{n}$ rate under an explicit parameter characterization. Moreover, we prove that polynomial partitioning estimates achieve the 1-dimensional min-max rate for regression of Hölder functions when combined to any $\sqrt{n}$-convergent index estimator. Overall this yields an estimator for dimension reduction and regression of single-index models that attains statistical optimality in quasilinear time.