Electronic Journal of Statistics

A Bayesian approach for noisy matrix completion: Optimal rate under general sampling distribution

The Tien Mai and Pierre Alquier

Full-text: Open access

Abstract

Bayesian methods for low-rank matrix completion with noise have been shown to be very efficient computationally [3, 18, 19, 24, 28]. While the behaviour of penalized minimization methods is well understood both from the theoretical and computational points of view (see [7, 9, 16, 23] among others) in this problem, the theoretical optimality of Bayesian estimators have not been explored yet. In this paper, we propose a Bayesian estimator for matrix completion under general sampling distribution. We also provide an oracle inequality for this estimator. This inequality proves that, whatever the rank of the matrix to be estimated, our estimator reaches the minimax-optimal rate of convergence (up to a logarithmic factor). We end the paper with a short simulation study.

Article information

Source
Electron. J. Statist. Volume 9, Number 1 (2015), 823-841.

Dates
First available in Project Euclid: 2 April 2015

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1427990076

Digital Object Identifier
doi:10.1214/15-EJS1020

Mathematical Reviews number (MathSciNet)
MR3331862

Zentralblatt MATH identifier
1317.62050

Subjects
Primary: 62H12: Estimation
Secondary: 62J12: Generalized linear models 60B20: Random matrices (probabilistic aspects; for algebraic aspects see 15B52) 68T05: Learning and adaptive systems [See also 68Q32, 91E40]

Keywords
Matrix completion Bayesian Analysis PAC-Bayesian bounds oracle inequality low-rank matrix Gibbs sampler

Citation

Mai, The Tien; Alquier, Pierre. A Bayesian approach for noisy matrix completion: Optimal rate under general sampling distribution. Electron. J. Statist. 9 (2015), no. 1, 823--841. doi:10.1214/15-EJS1020. https://projecteuclid.org/euclid.ejs/1427990076


Export citation

References

  • [1] Alquier, P., Bayesian methods for low-rank matrix estimation: Short survey and theoretical study. In, Algorithmic Learning Theory 2013, pages 309–323. Springer, 2013.
  • [2] Alquier, P. and Biau, G., Sparse single-index model., The Journal of Machine Learning Research, 14(1):243–280, 2013.
  • [3] Alquier, P., Cottet, V., Chopin, N., and Rousseau, J., Bayesian matrix completion: Prior specification., arXiv :1406.1440, 2014.
  • [4] Alquier, P. and Lounici, K., Pac-Bayesian bounds for sparse regression estimation with exponential weights., Electronic Journal of Statistics, 5:127–145, 2011.
  • [5] Bennett, J. and Lanning, S., The netflix prize. In, Proceedings of KDD Cup and Workshop, volume 2007, page 35, 2007.
  • [6] Boucheron, S., Lugosi, G., and Massart, P., Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, 2013.
  • [7] Candès, E. J. and Plan, Y., Matrix completion with noise., Proceedings of the IEEE, 98(6):925–936, 2010.
  • [8] Candès, E. J. and Recht, B., Exact matrix completion via convex optimization., Found. Comput. Math., 9(6):717–772, 2009.
  • [9] Candès, E. J. and Tao, T., The power of convex relaxation: Near-optimal matrix completion., IEEE Trans. Inform. Theory, 56(5) :2053–2080, 2010.
  • [10] Catoni, O., A PAC-Bayesian Approach to Adaptive Classification. Preprint Laboratoire de Probabilités et Modèles Aléatoires PMA-840, 2003.
  • [11] Catoni, O., Statistical Learning Theory and Stochastic Optimization. Saint-Flour Summer School on Probability Theory 2001 (Jean Picard ed.), Lecture Notes in Mathematics. Springer, 2004.
  • [12] Catoni, O., PAC-Bayesian Supervised Classification: The Thermodynamics of Statistical Learning. Institute of Mathematical Statistics Lecture Notes—Monograph Series, 56. Institute of Mathematical Statistics, Beachwood, OH, 2007.
  • [13] Dalalyan, A. and Tsybakov, A. B., Aggregation by exponential weighting, sharp pac-bayesian bounds and sparsity., Machine Learning, 72(1–2):39–61, 2008.
  • [14] Foygel, R., Shamir, O., Srebro, N., and Salakhutdinov, R., Learning with the weighted trace-norm under arbitrary sampling distributions. In, Advances in Neural Information Processing Systems, pages 2133–2141, 2011.
  • [15] Klopp, O., Noisy low-rank matrix completion with general sampling distribution., Bernoulli, 20(1):282–303, 2014.
  • [16] Koltchinskii, V., Lounici, K., and Tsybakov, A. B., Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion., The Annals of Statistics, 39(5) :2302–2329, 2011.
  • [17] Kotecha, J. H. and Djuric, P. M., Gibbs sampling approach for generation of truncated multivariate Gaussian random variables., Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing, 3 :1757–1760, 1999.
  • [18] Lawrence, N. D. and Urtasun, R., Non-linear matrix factorization with gaussian processes. In, Proceedings of the 26th Annual International Conference on Machine Learning, pages 601–608. ACM, 2009.
  • [19] Lim, Y. J. and Teh, Y. W., Variational bayesian approach to movie rating prediction. In, Proceedings of KDD Cup and Workshop, volume 7, pages 15–21, 2007.
  • [20] Massart, P., Concentration Inequalities and Model Selection, volume 1896 of Lecture Notes in Mathematics. Springer, Berlin, 2007. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6–23, 2003, Edited by Jean Picard.
  • [21] McAllester, D., Some PAC-Bayesian theorems. In, Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pages 230–234, New York, 1998. ACM.
  • [22] Negahban, S. and Wainwright, M. J., Restricted strong convexity and weighted matrix completion: Optimal bounds with noise., The Journal of Machine Learning Research, 13(1) :1665–1697, 2012.
  • [23] Recht, B. and Ré, C., Parallel stochastic gradient algorithms for large-scale matrix completion., Mathematical Programming Computation, 5(2):201–226, 2013.
  • [24] Salakhutdinov, R. and Mnih, A., Bayesian probabilistic matrix factorization using Markov Chain Monte Carlo. In, Proceedings of the 25th International Conference on Machine Learning, pages 880–887. ACM, 2008.
  • [25] Shawe-Taylor, J. and Williamson, R., A PAC analysis of a Bayes estimator. In, Proceedings of the Tenth Annual Conference on Computational Learning Theory, pages 2–9, New York, 1997. ACM.
  • [26] Suzuki, T., Convergence rate of bayesian tensor estimation: optimal rate without restricted strong convexity. arXiv, :1408.3092.
  • [27] Wilhelm, S., Package “tmvtnorm”, http://cran.r-project.org/web/packages/tmvtnorm/.
  • [28] Zhou, M., Wang, C., Chen, M., Paisley, J., Dunson, D., and Carin, L., Nonparametric bayesian matrix completion., Proc. IEEE SAM, 2010.