December 2024 Noisy recovery from random linear observations: Sharp minimax rates under elliptical constraints
Reese Pathak, Martin J. Wainwright, Lin Xiao
Author Affiliations +
Ann. Statist. 52(6): 2816-2850 (December 2024). DOI: 10.1214/24-AOS2446

Abstract

Estimation problems with constrained parameter spaces arise in various settings. In many of these problems, the observations available to the statistician can be modelled as arising from the noisy realization of the image of a random linear operator; an important special case is random design regression. We derive sharp rates of estimation for arbitrary compact elliptical parameter sets and demonstrate how they depend on the distribution of the random linear operator. Our main result is a functional that characterizes the minimax rate of estimation in terms of the noise level, the law of the random operator, and elliptical norms that define the error metric and the parameter space. This nonasymptotic result is sharp up to an explicit universal constant, and it becomes asymptotically exact as the radius of the parameter space is allowed to grow. We demonstrate the generality of the result by applying it to both parametric and nonparametric regression problems.

Funding Statement

RP was partially supported by a UC Berkeley Chancellor’s Fellowship via the ARCS Foundation.
MJW and RP were partially funded by ONR Grant N00014-21-1-2842, National Science Foundation Grant CCF-1955450, and National Science Foundation Grant DMS-2015454.
MJW and RP gratefully acknowledge funding support from Meta via the UC Berkeley AI Research (BAIR) Commons initiative.

Acknowledgments

We thank Jaouad Mourtada for a helpful conversation and useful email exchanges; we also thank Peter Bickel for a helpful discussion regarding his prior work on the Gaussian sequence model.

Citation

Download Citation

Reese Pathak. Martin J. Wainwright. Lin Xiao. "Noisy recovery from random linear observations: Sharp minimax rates under elliptical constraints." Ann. Statist. 52 (6) 2816 - 2850, December 2024. https://doi.org/10.1214/24-AOS2446

Information

Received: 1 March 2023; Revised: 1 March 2024; Published: December 2024
First available in Project Euclid: 18 December 2024

Digital Object Identifier: 10.1214/24-AOS2446

Subjects:
Primary: 62C20
Secondary: 62G08

Keywords: covariate shift , Minimax rates , Nonparametric regression , random design

Rights: Copyright © 2024 Institute of Mathematical Statistics

Vol.52 • No. 6 • December 2024
Back to Top