Abstract
We prove deviation inequalities for sums of high-dimensional random matrices and operators with dependence and heavy tails. Estimation of high-dimensional matrices is a concern for numerous modern applications. However, most results are stated for independent observations. Therefore, it is critical to derive results for dependent and heavy-tailed matrices. In this paper, we derive a dimension-free upper bound on the deviation of the sums. Thus, the bound does not depend explicitly on the dimension of the matrices but rather on their effective rank. Our result generalizes several existing studies on the deviation of sums of matrices. It relies on two techniques: (i) a variational approximation of the dual of moment generating functions, and (ii) robustification through the truncation of the eigenvalues of the matrices. We reveal that our results are applicable to several problems, such as covariance matrix estimation, hidden Markov models, and overparameterized linear regression.
Funding Statement
The first author’s work is supported by Japan Science and Technology Agency CREST (JPMJCR21D2). The second author’s work is partially funded by CY Initiative of Excellence (grant “Investissements d’Avenir” ANR-16-IDEX-0008), Project “EcoDep” PSI-AAP2020-0000000013. The third author’s work is supported by Japan Society for the Promotion of Science KAKENHI (21K11780) and Japan Science and Technology Agency FOREST (JPMJFR216I).
Citation
Shogo Nakakita. Pierre Alquier. Masaaki Imaizumi. "Dimension-free bounds for sums of dependent matrices and operators with heavy-tailed distributions." Electron. J. Statist. 18 (1) 1130 - 1159, 2024. https://doi.org/10.1214/24-EJS2224
Information