Translator Disclaimer
August 2022 Degrees of freedom for off-the-grid sparse estimation
Clarice Poon, Gabriel Peyré
Author Affiliations +
Bernoulli 28(3): 2095-2121 (August 2022). DOI: 10.3150/21-BEJ1414

Abstract

The degrees of freedom quantify the number of parameters of an estimator and is central to various risk minimization procedures. Its computation and properties are well understood when dealing with finite dimensional linear models, possibly regularized using sparsity through the celebrated Lasso estimator. However, for some applications (such as super-resolution in imaging or training multi-layer perceptrons with a single hidden layer) it makes sense to rather consider “continuous” methods. For these applications, these methods avoid the discretization of the parameter space and lead to more efficient numerical solvers. Training these continuous models with a sparsity inducing prior can be achieved by solving a convex optimization problem over the infinite dimensional space of measures, which is often called the Beurling Lasso (Blasso), and is the continuous counterpart of the Lasso. Previous works (Ann. Statist. 35 (2007) 2173–2192; Statist. Sinica 23 (2013) 809–828) show that the size of the smallest solution support is an unbiased estimator of the degrees of freedom of the Lasso. Our main contribution is a proof of a continuous counterpart to this result for the Blasso. In contrast to the Lasso, our new formula shows that the empirical degrees of freedom of the Blasso is not proportional to the size of the support. Our second contribution is a detailed study of the case of sampling Fourier coefficients in 1D, which corresponds to a super-resolution problem. We show that our formula for the unbiased estimation of the degrees of freedom is valid outside of a set of measure zero of observations, which in turn justifies its use to compute an unbiased estimator of the prediction risk using the Stein Unbiased Risk Estimator (SURE). We also report numerical results for both the case of Fourier sampling and the learning of a multilayers perceptron with a single hidden layer.

Citation

Download Citation

Clarice Poon. Gabriel Peyré. "Degrees of freedom for off-the-grid sparse estimation." Bernoulli 28 (3) 2095 - 2121, August 2022. https://doi.org/10.3150/21-BEJ1414

Information

Received: 1 December 2020; Published: August 2022
First available in Project Euclid: 25 April 2022

Digital Object Identifier: 10.3150/21-BEJ1414

Keywords: Beurling Lasso , Degrees of freedom , Lasso , off-the-grid

JOURNAL ARTICLE
27 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

SHARE
Vol.28 • No. 3 • August 2022
Back to Top