Electronic Journal of Statistics

Matrix completion by singular value thresholding: Sharp bounds

Olga Klopp

Full-text: Open access

Abstract

We consider the matrix completion problem where the aim is to estimate a large data matrix for which only a relatively small random subset of its entries is observed. Quite popular approaches to matrix completion problem are iterative thresholding methods. In spite of their empirical success, the theoretical guarantees of such iterative thresholding methods are poorly understood. The goal of this paper is to provide strong theoretical guarantees, similar to those obtained for nuclear-norm penalization methods and one step thresholding methods, for an iterative thresholding algorithm which is a modification of the softImpute algorithm. An important consequence of our result is the exact minimax optimal rates of convergence for matrix completion problem which were know until now only up to a logarithmic factor.

Article information

Source
Electron. J. Statist., Volume 9, Number 2 (2015), 2348-2369.

Dates
Received: January 2015
First available in Project Euclid: 23 October 2015

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1445605702

Digital Object Identifier
doi:10.1214/15-EJS1076

Mathematical Reviews number (MathSciNet)
MR3417185

Zentralblatt MATH identifier
1323.62047

Subjects
Primary: 62J99: None of the above, but in this section 62H12: Estimation 60B20: Random matrices (probabilistic aspects; for algebraic aspects see 15B52) 15A83: Matrix completion problems

Keywords
Matrix completion low rank matrix estimation minimax optimality

Citation

Klopp, Olga. Matrix completion by singular value thresholding: Sharp bounds. Electron. J. Statist. 9 (2015), no. 2, 2348--2369. doi:10.1214/15-EJS1076. https://projecteuclid.org/euclid.ejs/1445605702


Export citation

References

  • [1] Balzano, L., Nowak, R., and Recht, B. Absolute and monotonic norms., Numerische Mathematik, 3:257–264, 1961.
  • [2] Bandeira, A. S. and van Handel, R. Sharp nonasymptotic bounds on the norm of random matrices with independent entries., ArXiv e-prints, August 2014.
  • [3] Cai, J., Candès, E. J., and Shen Z. A singular value thresholding algorithm for matrix completion., SIAM Journal on Optimization, 20(4) :1956–1982, 2010.
  • [4] Cai, T. T. and Zhou, W. Matrix completion via max-norm constrained optimization., http://dx.doi.org/10.1007/978-1-4612-0537-1, 2013.
  • [5] Candès, E. J. and Plan, Y. Matrix completion with noise., Proceedings of IEEE, 98(6):925–936, 2009.
  • [6] Candès E. J. and Tao, T. The power of convex relaxation: near-optimal matrix completion., IEEE Trans. Inform. Theory, 56(5) :2053–2080, 2010.
  • [7] Candès, E. J. and Recht, B. Exact matrix completion via convex optimization., Fondations of Computational Mathematics, 9(6):717–772, 2009.
  • [8] Carpentier, A. and Kim, A. K. H. An iterative hard thresholding estimator for low rank matrix recovery with explicit limiting distribution., http://arxiv.org/abs/1502.04654, 2015.
  • [9] Chatterjee, S. Matrix estimation by universal singular value thresholding., Ann. Statist., 43(1):177–214, 02 2015.
  • [10] Dhanjal, C., Gaudel, R., and Clémençon, S. Online matrix completion through nuclear norm regularisation., SIAM International Conference on Data Mining, pages 623–631, 2014.
  • [11] Foygel, R. and Srebro, N. Concentration-based guarantees for low-rank matrix reconstruction., Journal 24nd Annual Conference on Learning Theory (COLT), 2011.
  • [12] Gross, D. Recovering low-rank matrices from few coefficients in any basis., IEEE Trans. Inform. Theory, 57(3) :1548–1566, 2011.
  • [13] Keshavan, R. H., Montanari, A. and Oh, S. Matrix completion from a few entries., IEEE Trans. Inform. Theory, 56(6) :2980–2998, 2010.
  • [14] Keshavan, R. H., Montanari, A., and Oh, S. Matrix completion from noisy entries., J. Mach. Learn. Res., 11 :2057–2078, 2010.
  • [15] Klopp, O. Rank penalized estimators for high-dimensional matrices., Electron. J. Statist., 5 :1161–1183, 2011.
  • [16] Klopp, O. Noisy low-rank matrix completion with general sampling distribution., Bernoulli, 20(1):282–303, 2014.
  • [17] Koltchinskii, V., Oracle inequalities in empirical risk minimization and sparse recovery problems, volume 2033 of Lecture Notes in Mathematics. Springer, Heidelberg, 2011. Lectures from the 38th Probability Summer School held in Saint-Flour, 2008, École d’Été de Probabilités de Saint-Flour. [Saint-Flour Probability Summer School].
  • [18] Koltchinskii, V., Lounici, K., and Tsybakov, A. B. Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion., Ann. Statist., 39(5) :2302–2329, 2011.
  • [19] Ledoux, M., The concentration of measure phenomenon, volume 89 of Mathematical Surveys and Monographs. American Mathematical Society, Providence, RI, 2001.
  • [20] Mazumder, R., Hastie, T., and Tibshirani, R. Spectral regularization algorithms for learning large incomplete matrices., Journal of Machine Learning Research, 11 :2287–2322, 2010.
  • [21] Negahban, S. and Wainwright, M. J. Restricted strong convexity and weighted matrix completion: Optimal bounds with noise., Journal of Machine Learning Research, 13 :1665–1697, 2012.
  • [22] Recht, B. A simpler approach to matrix completion., Journal of Machine Learning Research, 12 :3413–3430, 2011.
  • [23] Talagrand, M. A new look at independence., Ann. Probab., 24(1):1–34, 01 1996.
  • [24] Troyanskaya, O., Cantor, M., Sherlock, G., Brown, P., Hastie, T., Tibshirani, R., Botstein, D., and Altman, R. B. Missing value estimation methods for dna microarrays., Bioinformatics, 17(6):520–525, 2001.
  • [25] Tsybakov, A. B., Introduction to nonparametric estimation. Springer Series in Statistics. Springer, New York, 2009. Revised and extended from the 2004 French original, Translated by Vladimir Zaiats.
  • [26] Wang, Z., Lai, M., Lu, Z., Fan, W., Davulcu, H., and Ye, J. Orthogonal rank-one matrix pursuit for low rank matrix completion., Proceedings of the 31st International Conference on Machine Learning (ICML-14), pages 91–99, 2014.
  • [27] Watson, G. A. Characterization of the subdifferential of some matrix norms., Linear Algebra and its Applications, 170(0):33 – 45, 1992.