The Annals of Statistics
- Ann. Statist.
- Volume 39, Number 2 (2011), 1069-1097.
Estimation of (near) low-rank matrices with noise and high-dimensional scaling
We study an instance of high-dimensional inference in which the goal is to estimate a matrix Θ∗∈ℝm1×m2 on the basis of N noisy observations. The unknown matrix Θ∗ is assumed to be either exactly low rank, or “near” low-rank, meaning that it can be well-approximated by a matrix with low rank. We consider a standard M-estimator based on regularization by the nuclear or trace norm over matrices, and analyze its performance under high-dimensional scaling. We define the notion of restricted strong convexity (RSC) for the loss function, and use it to derive nonasymptotic bounds on the Frobenius norm error that hold for a general class of noisy observation models, and apply to both exactly low-rank and approximately low rank matrices. We then illustrate consequences of this general theory for a number of specific matrix models, including low-rank multivariate or multi-task regression, system identification in vector autoregressive processes and recovery of low-rank matrices from random projections. These results involve nonasymptotic random matrix theory to establish that the RSC condition holds, and to determine an appropriate choice of regularization parameter. Simulation results show excellent agreement with the high-dimensional scaling of the error predicted by our theory.
Ann. Statist. Volume 39, Number 2 (2011), 1069-1097.
First available in Project Euclid: 9 May 2011
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Negahban, Sahand; Wainwright, Martin J. Estimation of (near) low-rank matrices with noise and high-dimensional scaling. Ann. Statist. 39 (2011), no. 2, 1069--1097. doi:10.1214/10-AOS850. https://projecteuclid.org/euclid.aos/1304947044.
- Supplementary material: Supplement to “Estimation of (Near) Low-Rank Matrices with Noise and High-Dimensional Scaling”. Owing to space constraints, we have moved many of the technical proofs and details to the Appendix, which is contained in the supplementary document .