## The Annals of Statistics

- Ann. Statist.
- Volume 46, Number 1 (2018), 30-59.

### Optimal bounds for aggregation of affine estimators

#### Abstract

We study the problem of aggregation of estimators when the estimators are not independent of the data used for aggregation and no sample splitting is allowed. If the estimators are deterministic vectors, it is well known that the minimax rate of aggregation is of order $\log(M)$, where $M$ is the number of estimators to aggregate. It is proved that for affine estimators, the minimax rate of aggregation is unchanged: it is possible to handle the linear dependence between the affine estimators and the data used for aggregation at no extra cost. The minimax rate is not impacted either by the variance of the affine estimators, or any other measure of their statistical complexity. The minimax rate is attained with a penalized procedure over the convex hull of the estimators, for a penalty that is inspired from the $Q$-aggregation procedure. The results follow from the interplay between the penalty, strong convexity and concentration.

#### Article information

**Source**

Ann. Statist., Volume 46, Number 1 (2018), 30-59.

**Dates**

Received: September 2015

Revised: December 2016

First available in Project Euclid: 22 February 2018

**Permanent link to this document**

https://projecteuclid.org/euclid.aos/1519268423

**Digital Object Identifier**

doi:10.1214/17-AOS1540

**Mathematical Reviews number (MathSciNet)**

MR3766945

**Zentralblatt MATH identifier**

06865104

**Subjects**

Primary: 62G05: Estimation

Secondary: 62J07: Ridge regression; shrinkage estimators

**Keywords**

Affine estimator aggregation sequence model sharp oracle inequality concentration inequality Hanson–Wright

#### Citation

Bellec, Pierre C. Optimal bounds for aggregation of affine estimators. Ann. Statist. 46 (2018), no. 1, 30--59. doi:10.1214/17-AOS1540. https://projecteuclid.org/euclid.aos/1519268423