Open Access
Translator Disclaimer
April 2018 Sharp oracle inequalities for Least Squares estimators in shape restricted regression
Pierre C. Bellec
Ann. Statist. 46(2): 745-780 (April 2018). DOI: 10.1214/17-AOS1566

Abstract

The performance of Least Squares (LS) estimators is studied in shape-constrained regression models under Gaussian and sub-Gaussian noise. General bounds on the performance of LS estimators over closed convex sets are provided. These results have the form of sharp oracle inequalities that account for the model misspecification error. In the presence of misspecification, these bounds imply that the LS estimator estimates the projection of the true parameter at the same rate as in the well-specified case.

In isotonic and unimodal regression, the LS estimator achieves the nonparametric rate $n^{-2/3}$ as well as a parametric rate of order $k/n$ up to logarithmic factors, where $k$ is the number of constant pieces of the true parameter. In univariate convex regression, the LS estimator satisfies an adaptive risk bound of order $q/n$ up to logarithmic factors, where $q$ is the number of affine pieces of the true regression function. This adaptive risk bound holds for any collection of design points. While Guntuboyina and Sen [Probab. Theory Related Fields 163 (2015) 379–411] established that the nonparametric rate of convex regression is of order $n^{-4/5}$ for equispaced design points, we show that the nonparametric rate of convex regression can be as slow as $n^{-2/3}$ for some worst-case design points. This phenomenon can be explained as follows: Although convexity brings more structure than unimodality, for some worst-case design points this extra structure is uninformative and the nonparametric rates of unimodal regression and convex regression are both $n^{-2/3}$. Higher order cones, such as the cone of $\beta $-monotone sequences, are also studied.

Citation

Download Citation

Pierre C. Bellec. "Sharp oracle inequalities for Least Squares estimators in shape restricted regression." Ann. Statist. 46 (2) 745 - 780, April 2018. https://doi.org/10.1214/17-AOS1566

Information

Received: 1 October 2015; Revised: 1 March 2017; Published: April 2018
First available in Project Euclid: 3 April 2018

zbMATH: 06870278
MathSciNet: MR3782383
Digital Object Identifier: 10.1214/17-AOS1566

Subjects:
Primary: 62C20 , 62G08

Keywords: Concentration , convexity , Gaussian width , Minimax rates , shape restricted regression

Rights: Copyright © 2018 Institute of Mathematical Statistics

JOURNAL ARTICLE
36 PAGES


SHARE
Vol.46 • No. 2 • April 2018
Back to Top