The Annals of Statistics

Sparse recovery under matrix uncertainty

Mathieu Rosenbaum and Alexandre B. Tsybakov

Full-text: Open access

Abstract

We consider the model

y =  + ξ,

Z = X + Ξ,

where the random vector y ∈ ℝn and the random n × p matrix Z are observed, the n × p matrix X is unknown, Ξ is an n × p random noise matrix, ξ ∈ ℝn is a noise independent of Ξ, and θ is a vector of unknown parameters to be estimated. The matrix uncertainty is in the fact that X is observed with additive error. For dimensions p that can be much larger than the sample size n, we consider the estimation of sparse vectors θ. Under matrix uncertainty, the Lasso and Dantzig selector turn out to be extremely unstable in recovering the sparsity pattern (i.e., of the set of nonzero components of θ), even if the noise level is very small. We suggest new estimators called matrix uncertainty selectors (or, shortly, the MU-selectors) which are close to θ in different norms and in the prediction risk if the restricted eigenvalue assumption on X is satisfied. We also show that under somewhat stronger assumptions, these estimators recover correctly the sparsity pattern.

Article information

Source
Ann. Statist., Volume 38, Number 5 (2010), 2620-2651.

Dates
First available in Project Euclid: 11 July 2010

Permanent link to this document
https://projecteuclid.org/euclid.aos/1278861455

Digital Object Identifier
doi:10.1214/10-AOS793

Mathematical Reviews number (MathSciNet)
MR2722451

Zentralblatt MATH identifier
1373.62357

Subjects
Primary: 62J05: Linear regression
Secondary: 62F12: Asymptotic properties of estimators

Keywords
Sparsity MU-selector matrix uncertainty errors-in-variables model measurement error sign consistency oracle inequalities restricted eigenvalue assumption missing data portfolio selection portfolio replication

Citation

Rosenbaum, Mathieu; Tsybakov, Alexandre B. Sparse recovery under matrix uncertainty. Ann. Statist. 38 (2010), no. 5, 2620--2651. doi:10.1214/10-AOS793. https://projecteuclid.org/euclid.aos/1278861455


Export citation

References

  • [1] Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous analysis of Lasso and Dantzig selector. Ann. Statist. 37 1705–1732.
  • [2] Brodie, J., Daubechies, I., De Molc, C., Giannone, D. and Loris, I. (2009). Sparse and stable Markowitz portfolios. PNAS 106 12267–12272.
  • [3] Bunea, F., Tsybakov, A. B. and Wegkamp, M. H. (2007). Aggregation for Gaussian regression. Ann. Statist. 35 1674–1697.
  • [4] Bunea, F., Tsybakov, A. B. and Wegkamp, M. H. (2007). Sparsity oracle inequalities for the Lasso. Electron. J. Stat. 1 169–194.
  • [5] Candès, E. J. (2008). The restricted isometry property and its implications for compressed sensing. C. R. Math. Acad. Sci. Paris 346 589–592.
  • [6] Candès, E. J., Romberg, J. and Tao, T. (2005). Signal recovery from incomplete and inaccurate measurements. Comm. Pure Appl. Math. 59 1207–1223.
  • [7] Candès, E. J. and Tao, T. (2006). Decoding by linear programming. IEEE Trans. Inform. Theory 51 4203–4215.
  • [8] Candès, E. J. and Tao, T. (2007). The Dantzig selector: Statistical estimation when p is much larger than n (with discussion). Ann. Statist. 35 2313–2404.
  • [9] Cavalier, L. and Hengartner, N. W. (2005). Adaptive estimation for inverse problems with noisy operators. Inverse Problems 21 1345–1361.
  • [10] Cavalier, L. and Raimondo, M. (2007). Wavelet deconvolution with noisy eigen-values. IEEE Trans. Signal Process. 55 2414–2424.
  • [11] Donoho, D. L., Elad, M. and Temlyakov, V. (2006). Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inform. Theory 52 6–18.
  • [12] Draper, N. R. and Smith, H. (1998). Applied Regression Analysis. Wiley, New York.
  • [13] Dalalyan, A. and Tsybakov A. B. (2008). Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity. Mach. Learn. 72 39–61.
  • [14] Dalalyan, A. and Tsybakov A. B. (2009). Sparse regression learning by aggregation and Langevin Monte-Carlo. In Proceedings of COLT-2009. Available at arXiv:0903.1223.
  • [15] Efromovich, S. and Koltchinskii, V. (2001). On inverse problems with unknown operators. IEEE Trans. Inform. Theory 47 2876–2894.
  • [16] Fuller, W. A. (1987). Measurement Error Models. Wiley, New York.
  • [17] Hoffmann, M. and Reiss, M. (2008). Nonlinear estimation for linear inverse problems with error in the operator. Ann. Statist. 36 310–336.
  • [18] Lounici, K. (2008). Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators. Electron. J. Stat. 2 90–102.
  • [19] Lounici, K. (2010). High-dimensional stochastic optimization with the generalized Dantzig estimator. To appear. Available at arXiv:0811.2281v1.
  • [20] Koltchinskii, V. (2009). Dantzig selector and sparsity oracle inequalities. Bernoulli 15 799–828.
  • [21] Koltchinskii, V. (2010). Oracle inequalities in empirical risk minimization and sparse recovery problems. St Flour Lecture Notes.
  • [22] Marteau, C. (2007). Regularization of inverse problems with unknown operator. Math. Methods Statist. 15 415–443.
  • [23] Meinshausen, N. and Bühlmann, P. (2006). High dimensional graphs and variable selection with the Lasso. Ann. Statist. 34 1436–1462.
  • [24] van de Geer, S. A. (2008). High dimensional generalized linear models and the Lasso. Ann. Statist. 36 614–645.
  • [25] Zhang, C. H. and Huang, J. (2008). The sparsity and biais of the Lasso selection in high-dimensional linear regression. Ann. Statist. 36 1567–1594.
  • [26] Zhang, T. (2009). Some sharp performance bounds for least squares regression with L1 regularization. Ann. Statist. 37 2109–2144.
  • [27] Zhao, P. and Yu, B. (2007). On model selection consistency of Lasso. J. Mach. Learn. Res. 7 2541–2567.
  • [28] Zou, H. (2006). The adaptive Lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418–1429.