August 2023 Noisy linear inverse problems under convex constraints: Exact risk asymptotics in high dimensions
Qiyang Han
Author Affiliations +
Ann. Statist. 51(4): 1611-1638 (August 2023). DOI: 10.1214/23-AOS2301

Abstract

In the standard Gaussian linear measurement model Y=Xμ0+ξRm with a fixed noise level σ>0, we consider the problem of estimating the unknown signal μ0 under a convex constraint μ0K, where K is a closed convex set in Rn. We show that the risk of the natural convex constrained least squares estimator (LSE) μˆ(σ) can be characterized exactly in high-dimensional limits, by that of the convex constrained LSE μˆKseq in the corresponding Gaussian sequence model at a different noise level. Formally, we show that

μˆ(σ)μ02/(nrn2)1in probability,

where rn2>0 solves the fixed-point equation

EμˆKseq((rn2+σ2)/(m/n))μ02=nrn2.

This characterization holds (uniformly) for risks rn2 in the maximal regime that ranges from constant order all the way down to essentially the parametric rate, as long as certain necessary nondegeneracy condition is satisfied for μˆ(σ).

The precise risk characterization reveals a fundamental difference between noiseless (or low noise limit) and noisy linear inverse problems in terms of the sample complexity for signal recovery. A concrete example is given by the isotonic regression problem: While exact recovery of a general monotone signal requires mn1/3 samples in the noiseless setting, consistent signal recovery in the noisy setting requires as few as mlogn samples. Such a discrepancy occurs when the low and high noise risk behavior of μˆKseq differ significantly. In statistical languages, this occurs when μˆKseq estimates 0 at a faster “adaptation rate” than the slower “worst-case rate” for general signals. Several other examples, including nonnegative least squares and generalized Lasso (in constrained forms), are also worked out to demonstrate the concrete applicability of the theory in problems of different types.

The proof relies on a collection of new analytic and probabilistic results concerning estimation error, log likelihood ratio test statistics and degree-of-freedom associated with μˆKseq, regarded as stochastic processes indexed by the noise level. These results are of independent interest in and of themselves.

Funding Statement

The research of Q. Han is partially supported by NSF Grants DMS-1916221 and DMS-2143468.

Acknowledgments

The author would like to thank three referees, an Associate Editor and the Editor for a large number of helpful comments and suggestions that significantly improved the quality of the paper.

Citation

Download Citation

Qiyang Han. "Noisy linear inverse problems under convex constraints: Exact risk asymptotics in high dimensions." Ann. Statist. 51 (4) 1611 - 1638, August 2023. https://doi.org/10.1214/23-AOS2301

Information

Received: 1 February 2022; Revised: 1 May 2023; Published: August 2023
First available in Project Euclid: 19 October 2023

Digital Object Identifier: 10.1214/23-AOS2301

Subjects:
Primary: 60F05 , 62G08
Secondary: 62E17 , 62G10

Keywords: fixed-point equation , Gaussian sequence model , high-dimensional asymptotics , Linear inverse problem

Rights: Copyright © 2023 Institute of Mathematical Statistics

JOURNAL ARTICLE
28 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

Vol.51 • No. 4 • August 2023
Back to Top