Matrix factorization is a powerful data analysis tool. It has been used in multivariate time series analysis, leading to the decomposition of the series in a small set of latent factors. However, little is known on the statistical performances of matrix factorization for time series. In this paper, we extend the results known for matrix estimation in the i.i.d setting to time series. Moreover, we prove that when the series exhibit some additional structure like periodicity or smoothness, it is possible to improve on the classical rates of convergence.
References
[1] P. Alquier. Bayesian methods for low-rank matrix estimation: short survey and theoretical study. In, International Conference on Algorithmic Learning Theory, pages 309–323. Springer, 2013. 1411.62136[1] P. Alquier. Bayesian methods for low-rank matrix estimation: short survey and theoretical study. In, International Conference on Algorithmic Learning Theory, pages 309–323. Springer, 2013. 1411.62136
[2] P. Alquier, V. Cottet, and G. Lecué. Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions., arXiv preprint, to appear in the Annals of Statistics, 2017.[2] P. Alquier, V. Cottet, and G. Lecué. Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions., arXiv preprint, to appear in the Annals of Statistics, 2017.
[3] P. Alquier and P. Doukhan. Sparsity considerations for dependent variables., Electronic journal of statistics, 5:750–774, 2011. 1274.62462 10.1214/11-EJS626[3] P. Alquier and P. Doukhan. Sparsity considerations for dependent variables., Electronic journal of statistics, 5:750–774, 2011. 1274.62462 10.1214/11-EJS626
[4] P. Alquier and B. Guedj. An oracle inequality for quasi-Bayesian nonnegative matrix factorization., Mathematical Methods of Statistics, 26(1):55–67, 2017. 1381.62222 10.3103/S1066530717010045[4] P. Alquier and B. Guedj. An oracle inequality for quasi-Bayesian nonnegative matrix factorization., Mathematical Methods of Statistics, 26(1):55–67, 2017. 1381.62222 10.3103/S1066530717010045
[5] L. Bauwens and M. Lubrano. Identification restriction and posterior densities in cointegrated Gaussian VAR systems. In T. M. Fomby and R. Carter Hill, editors, Advances in econometrics, vol. 11(B). JAI Press, Greenwich, 1993.[5] L. Bauwens and M. Lubrano. Identification restriction and posterior densities in cointegrated Gaussian VAR systems. In T. M. Fomby and R. Carter Hill, editors, Advances in econometrics, vol. 11(B). JAI Press, Greenwich, 1993.
[6] S. Boucheron, G. Lugosi, and P. Massart., Concentration inequalities: A nonasymptotic theory of independence. Oxford university press, 2013. 1279.60005[6] S. Boucheron, G. Lugosi, and P. Massart., Concentration inequalities: A nonasymptotic theory of independence. Oxford university press, 2013. 1279.60005
[7] T. Cai, D. Kim, Y. Wang, M. Yuan, and H. Zhou. Optimal large-scale quantum state tomography with Pauli measurements., The Annals of Statistics, 44(2):682–712, 2016. 1341.62116 10.1214/15-AOS1382 euclid.aos/1458245732[7] T. Cai, D. Kim, Y. Wang, M. Yuan, and H. Zhou. Optimal large-scale quantum state tomography with Pauli measurements., The Annals of Statistics, 44(2):682–712, 2016. 1341.62116 10.1214/15-AOS1382 euclid.aos/1458245732
[8] T. Cai and A. Zhang. Rop: Matrix recovery via rank-one projections., The Annals of Statistics, 43(1):102–138, 2015. 1308.62120 10.1214/14-AOS1267 euclid.aos/1416322038[8] T. Cai and A. Zhang. Rop: Matrix recovery via rank-one projections., The Annals of Statistics, 43(1):102–138, 2015. 1308.62120 10.1214/14-AOS1267 euclid.aos/1416322038
[11] E. J. Candès and T. Tao. The power of convex relaxation: Near-optimal matrix completion., IEEE Transactions on Information Theory, 56(5) :2053–2080, 2010. 1366.15021 10.1109/TIT.2010.2044061[11] E. J. Candès and T. Tao. The power of convex relaxation: Near-optimal matrix completion., IEEE Transactions on Information Theory, 56(5) :2053–2080, 2010. 1366.15021 10.1109/TIT.2010.2044061
[12] L. Carel and P. Alquier. Non-negative matrix factorization as a pre-processing tool for travelers temporal profiles clustering. In, Proceedings of the 25th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pages 417–422, 2017.[12] L. Carel and P. Alquier. Non-negative matrix factorization as a pre-processing tool for travelers temporal profiles clustering. In, Proceedings of the 25th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pages 417–422, 2017.
[13] D. Chafaï, O. Guédon, G. Lecué, and A. Pajor., Interactions between compressed sensing random matrices and high dimensional geometry. Société Mathématique de France, 2012.[13] D. Chafaï, O. Guédon, G. Lecué, and A. Pajor., Interactions between compressed sensing random matrices and high dimensional geometry. Société Mathématique de France, 2012.
[14] V. Cheung, K. Devarajan, G. Severini, A. Turolla, and P. Bonato. Decomposing time series data by a non-negative matrix factorization algorithm with temporally constrained coefficients. In, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 3496–3499. IEEE, 2015.[14] V. Cheung, K. Devarajan, G. Severini, A. Turolla, and P. Bonato. Decomposing time series data by a non-negative matrix factorization algorithm with temporally constrained coefficients. In, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 3496–3499. IEEE, 2015.
[17] A. S. Dalalyan, E. Grappin, and Q. Paris. On the exponentially weighted aggregate with the Laplace prior., The Annals of Statistics, 46(5) :2452–2478, 2018. 1409.62135 10.1214/17-AOS1626 euclid.aos/1534492841[17] A. S. Dalalyan, E. Grappin, and Q. Paris. On the exponentially weighted aggregate with the Laplace prior., The Annals of Statistics, 46(5) :2452–2478, 2018. 1409.62135 10.1214/17-AOS1626 euclid.aos/1534492841
[18] Y. De Castro, Y. Goude, G. Hébrail, and J. Mei. Recovering multiple nonnegative time series from a few temporal aggregates. In, ICML 2017-34th International Conference on Machine Learning, pages 1–9, 2017.[18] Y. De Castro, Y. Goude, G. Hébrail, and J. Mei. Recovering multiple nonnegative time series from a few temporal aggregates. In, ICML 2017-34th International Conference on Machine Learning, pages 1–9, 2017.
[19] J. Dedecker, P. Doukhan, G. Lang, L. R. J. Rafael, S. Louhichi, and C. Prieur., Weak dependence: With examples and applications. Springer, 2007. 1165.62001[19] J. Dedecker, P. Doukhan, G. Lang, L. R. J. Rafael, S. Louhichi, and C. Prieur., Weak dependence: With examples and applications. Springer, 2007. 1165.62001
[20] R. F. Engle and C. W. J. Granger. Co-integration and error correction: representation, estimation, and testing., Econometrica: journal of the Econometric Society, pages 251–276, 1987. 0613.62140 10.2307/1913236[20] R. F. Engle and C. W. J. Granger. Co-integration and error correction: representation, estimation, and testing., Econometrica: journal of the Econometric Society, pages 251–276, 1987. 0613.62140 10.2307/1913236
[21] I. A. Genevera, L. Grosenick, and J. Taylor. A generalized least-square matrix decomposition., Journal of the American Statistical Association, 109(505):145–159, 2014. 1367.62184 10.1080/01621459.2013.852978[21] I. A. Genevera, L. Grosenick, and J. Taylor. A generalized least-square matrix decomposition., Journal of the American Statistical Association, 109(505):145–159, 2014. 1367.62184 10.1080/01621459.2013.852978
[22] J. Geweke. Bayesian reduced rank regression in econometrics., Journal of Econometrics, 75:121–146, 1996. MR1414507 0864.62083 10.1016/0304-4076(95)01773-9[22] J. Geweke. Bayesian reduced rank regression in econometrics., Journal of Econometrics, 75:121–146, 1996. MR1414507 0864.62083 10.1016/0304-4076(95)01773-9
[23] D. Gross. Recovering low-rank matrices from few coefficients in any basis., Information Theory, IEEE Transactions on, 57(3) :1548–1566, 2011. 1366.94103 10.1109/TIT.2011.2104999[23] D. Gross. Recovering low-rank matrices from few coefficients in any basis., Information Theory, IEEE Transactions on, 57(3) :1548–1566, 2011. 1366.94103 10.1109/TIT.2011.2104999
[24] S. Gultekin and J. Paisley. Online forecasting matrix factorization., arXiv preprint arXiv :1712.08734, 2017. 1414.62393 10.1109/TSP.2018.2889982[24] S. Gultekin and J. Paisley. Online forecasting matrix factorization., arXiv preprint arXiv :1712.08734, 2017. 1414.62393 10.1109/TSP.2018.2889982
[25] M. Guţă, T. Kypraios, and I. Dryden. Rank-based model selection for multiple ions quantum tomography., New Journal of Physics, 14(10) :105002, 2012.[25] M. Guţă, T. Kypraios, and I. Dryden. Rank-based model selection for multiple ions quantum tomography., New Journal of Physics, 14(10) :105002, 2012.
[26] F; Husson, J. Josse, B. Narasimhan, and G. Robin. Imputation of mixed data with multilevel singular value decomposition., arXiv preprint arXiv :1804.11087, 2018.[26] F; Husson, J. Josse, B. Narasimhan, and G. Robin. Imputation of mixed data with multilevel singular value decomposition., arXiv preprint arXiv :1804.11087, 2018.
[27] A. Izenman. Reduced rank regression for the multivariate linear model., Journal of Multivariate Analysis, 5(2):248–264, 1975. 0313.62042 10.1016/0047-259X(75)90042-1[27] A. Izenman. Reduced rank regression for the multivariate linear model., Journal of Multivariate Analysis, 5(2):248–264, 1975. 0313.62042 10.1016/0047-259X(75)90042-1
[30] O. Klopp, K. Lounici, and A. B. Tsybakov. Robust matrix completion., Probability Theory and Related Fields, 169(1–2):523–564, 2017. 1383.62167 10.1007/s00440-016-0736-y[30] O. Klopp, K. Lounici, and A. B. Tsybakov. Robust matrix completion., Probability Theory and Related Fields, 169(1–2):523–564, 2017. 1383.62167 10.1007/s00440-016-0736-y
[31] O. Klopp, Y. Lu, A. B. Tsybakov, and H. H. Zhou. Structured matrix estimation and completion., arXiv preprint arXiv :1707.02090, 2017. 07110159 10.3150/19-BEJ1114 euclid.bj/1569398788[31] O. Klopp, Y. Lu, A. B. Tsybakov, and H. H. Zhou. Structured matrix estimation and completion., arXiv preprint arXiv :1707.02090, 2017. 07110159 10.3150/19-BEJ1114 euclid.bj/1569398788
[32] V. Koltchinskii, K. Lounici, and A. B. Tsybakov. Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion., The Annals of Statistics, 39(5) :2302–2329, 2011. 1231.62097 10.1214/11-AOS894 euclid.aos/1322663459[32] V. Koltchinskii, K. Lounici, and A. B. Tsybakov. Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion., The Annals of Statistics, 39(5) :2302–2329, 2011. 1231.62097 10.1214/11-AOS894 euclid.aos/1322663459
[33] G. Koop and D. Korobilis. Bayesian multivariate time series methods for empirical macroeconomics., Foundations and Trends® in Econometrics, 3(4):267–358, 2010. 1193.91117 10.1561/0800000013[33] G. Koop and D. Korobilis. Bayesian multivariate time series methods for empirical macroeconomics., Foundations and Trends® in Econometrics, 3(4):267–358, 2010. 1193.91117 10.1561/0800000013
[35] D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization., Nature, 401 (6755):788–791, 1999. 1369.68285 10.1038/44565[35] D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization., Nature, 401 (6755):788–791, 1999. 1369.68285 10.1038/44565
[36] A. Lumbreras, L. Filstroff, and C. Févotte. Bayesian mean-parameterized nonnegative binary matrix factorization., arXiv preprint arXiv :1812.06866, 2018.[36] A. Lumbreras, L. Filstroff, and C. Févotte. Bayesian mean-parameterized nonnegative binary matrix factorization., arXiv preprint arXiv :1812.06866, 2018.
[38] T. T. Mai and P. Alquier. A Bayesian approach for noisy matrix completion: Optimal rate under general sampling distribution., Electronic Journal of Statistics, 9(1):823–841, 2015. 1317.62050 10.1214/15-EJS1020[38] T. T. Mai and P. Alquier. A Bayesian approach for noisy matrix completion: Optimal rate under general sampling distribution., Electronic Journal of Statistics, 9(1):823–841, 2015. 1317.62050 10.1214/15-EJS1020
[39] T. T. Mai and P. Alquier. Pseudo-Bayesian quantum tomography with rank-adaptation., Journal of Statistical Planning and Inference, 184:62–76, 2017. 1395.62379 10.1016/j.jspi.2016.11.003[39] T. T. Mai and P. Alquier. Pseudo-Bayesian quantum tomography with rank-adaptation., Journal of Statistical Planning and Inference, 184:62–76, 2017. 1395.62379 10.1016/j.jspi.2016.11.003
[40] J. Mei, Y. De Castro, Y. Goude, J.-M. Azaïs, and G. Hébrail. Nonnegative matrix factorization with side information for time series recovery and prediction., IEEE Transactions on Knowledge and Data Engineering, 2018.[40] J. Mei, Y. De Castro, Y. Goude, J.-M. Azaïs, and G. Hébrail. Nonnegative matrix factorization with side information for time series recovery and prediction., IEEE Transactions on Knowledge and Data Engineering, 2018.
[41] K. Moridomi, K. Hatano, and E. Takimoto. Tighter generalization bounds for matrix completion via factorization into constrained matrices., IEICE Transactions on Information and Systems, 101(8) :1997–2004, 2018.[41] K. Moridomi, K. Hatano, and E. Takimoto. Tighter generalization bounds for matrix completion via factorization into constrained matrices., IEICE Transactions on Information and Systems, 101(8) :1997–2004, 2018.
[42] A. Ozerov and C. Févotte. Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation., IEEE Transactions on Audio, Speech, and Language Processing, 18(3):550–563, 2010.[42] A. Ozerov and C. Févotte. Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation., IEEE Transactions on Audio, Speech, and Language Processing, 18(3):550–563, 2010.
[43] J. Paisley, D. Blei, and M. I. Jordan., Bayesian nonnegative matrix factorization with stochastic variational inference, volume Handbook of Mixed Membership Models and Their Applications, chapter 11. Chapman and Hall/CRC, 2015.[43] J. Paisley, D. Blei, and M. I. Jordan., Bayesian nonnegative matrix factorization with stochastic variational inference, volume Handbook of Mixed Membership Models and Their Applications, chapter 11. Chapman and Hall/CRC, 2015.
[44] E. Richard, S. Gaïffas, and N. Vayatis. Link prediction in graphs with autoregressive features., The Journal of Machine Learning Research, 15(1):565–593, 2014. 1318.62183[44] E. Richard, S. Gaïffas, and N. Vayatis. Link prediction in graphs with autoregressive features., The Journal of Machine Learning Research, 15(1):565–593, 2014. 1318.62183
[45] A. Saha and V. Sindhwani. Learning evolving and emerging topics in social media: a dynamic nmf approach with temporal regularization. In, Proceedings of the fifth ACM international conference on Web search and data mining, pages 693–702. ACM, 2012.[45] A. Saha and V. Sindhwani. Learning evolving and emerging topics in social media: a dynamic nmf approach with temporal regularization. In, Proceedings of the fifth ACM international conference on Web search and data mining, pages 693–702. ACM, 2012.
[46] F. Shahnaz, M. W. Berry, V. P. Pauca, and R. J. Plemmons. Document clustering using nonnegative matrix factorization., Information Processing & Management, 42(2):373–386, 2006. 1087.68104 10.1016/j.ipm.2004.11.005[46] F. Shahnaz, M. W. Berry, V. P. Pauca, and R. J. Plemmons. Document clustering using nonnegative matrix factorization., Information Processing & Management, 42(2):373–386, 2006. 1087.68104 10.1016/j.ipm.2004.11.005
[47] T. Suzuki. Convergence rate of Bayesian tensor estimator and its minimax optimality. In, International Conference on Machine Learning, pages 1273–1282, 2015.[47] T. Suzuki. Convergence rate of Bayesian tensor estimator and its minimax optimality. In, International Conference on Machine Learning, pages 1273–1282, 2015.
[48] E. Tonnelier, N. Baskiotis, V. Guigue, and P. Gallinari. Anomaly detection in smart card logs and distant evaluation with twitter: a robust framework., Neurocomputing, 298:109–121, 2018.[48] E. Tonnelier, N. Baskiotis, V. Guigue, and P. Gallinari. Anomaly detection in smart card logs and distant evaluation with twitter: a robust framework., Neurocomputing, 298:109–121, 2018.
[49] J. A. Tropp. User-friendly tail bounds for sums of random matrices., Foundations of computational mathematics, 12(4):389–434, 2012. 1259.60008 10.1007/s10208-011-9099-z[49] J. A. Tropp. User-friendly tail bounds for sums of random matrices., Foundations of computational mathematics, 12(4):389–434, 2012. 1259.60008 10.1007/s10208-011-9099-z
[50] A. B. Tsybakov., Introduction to Nonparametric Estimation. 2009. 1176.62032[50] A. B. Tsybakov., Introduction to Nonparametric Estimation. 2009. 1176.62032
[51] C. Vernade and O. Cappé. Learning from missing data using selection bias in movie recommendation. In, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pages 1–9. IEEE, 2015.[51] C. Vernade and O. Cappé. Learning from missing data using selection bias in movie recommendation. In, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pages 1–9. IEEE, 2015.
[53] D. Xia and V. Koltchinskii. Estimation of low rank density matrices: bounds in schatten norms and other distances., Electronic Journal of Statistics, 10(2) :2717–2745, 2016. 1403.62104 10.1214/16-EJS1192[53] D. Xia and V. Koltchinskii. Estimation of low rank density matrices: bounds in schatten norms and other distances., Electronic Journal of Statistics, 10(2) :2717–2745, 2016. 1403.62104 10.1214/16-EJS1192
[54] H.-F. Yu, N. Rao, and I. S Dhillon. Temporal regularized matrix factorization for high-dimensional time series prediction. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 847–855. Curran Associates, Inc., 2016.[54] H.-F. Yu, N. Rao, and I. S Dhillon. Temporal regularized matrix factorization for high-dimensional time series prediction. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 847–855. Curran Associates, Inc., 2016.
