Mixing least-squares estimators when the variance is unknown

Abstract

We propose a procedure to handle the problem of Gaussian regression when the variance is unknown. We mix least-squares estimators from various models according to a procedure inspired by that of Leung and Barron [IEEE Trans. Inform. Theory 52 (2006) 3396–3410]. We show that in some cases, the resulting estimator is a simple shrinkage estimator. We then apply this procedure to perform adaptive estimation in Besov spaces. Our results provide non-asymptotic risk bounds for the Euclidean risk of the estimator.