Abstract
This paper is concerned with the asymptotic empirical eigenvalue distribution of some non linear random matrix ensemble. More precisely we consider with where W and X are random rectangular matrices with i.i.d. centered entries. The function f is applied pointwise and can be seen as an activation function in (random) neural networks. We compute the asymptotic empirical distribution of this ensemble in the case where W and X have sub-Gaussian tails and f is real analytic. This extends a result of [32] where the case of Gaussian matrices W and X is considered. We also investigate the same questions in the multi-layer case, regarding neural network and machine learning applications.
Funding Statement
Research was accomplished while Sandrine Péché was supported by the Institut Universitaire de France.
Acknowledgments
The authors would like to thank D. Schröder and Z. Fan for pointing out errors in a previous version of the article as well as anonymous referees for helpful suggestions on how to improve the present paper.
Citation
Lucas Benigni. Sandrine Péché. "Eigenvalue distribution of some nonlinear models of random matrices." Electron. J. Probab. 26 1 - 37, 2021. https://doi.org/10.1214/21-EJP699
Information