Open Access
Translator Disclaimer
2022 Convergence rates of deep ReLU networks for multiclass classification
Thijs Bos, Johannes Schmidt-Hieber
Author Affiliations +
Electron. J. Statist. 16(1): 2724-2773 (2022). DOI: 10.1214/22-EJS2011

Abstract

For classification problems, trained deep neural networks return probabilities of class memberships. In this work we study convergence of the learned probabilities to the true conditional class probabilities. More specifically we consider sparse deep ReLU network reconstructions minimizing cross-entropy loss in the multiclass classification setup. Interesting phenomena occur when the class membership probabilities are close to zero. Convergence rates are derived that depend on the near-zero behaviour via a margin-type condition.

Funding Statement

The research has been supported by the NWO/STAR grant 613.009.034b and the NWO Vidi grant VI.Vidi.192.021.

Acknowledgments

We thank the two referees and the editor for many helpful suggestions.

Citation

Download Citation

Thijs Bos. Johannes Schmidt-Hieber. "Convergence rates of deep ReLU networks for multiclass classification." Electron. J. Statist. 16 (1) 2724 - 2773, 2022. https://doi.org/10.1214/22-EJS2011

Information

Received: 1 August 2021; Published: 2022
First available in Project Euclid: 12 April 2022

Digital Object Identifier: 10.1214/22-EJS2011

Subjects:
Primary: 62G05
Secondary: 63H30 , 68T07

Keywords: conditional class probabilities , Convergence rates , margin condition , Multiclass classification , ReLU networks

JOURNAL ARTICLE
50 PAGES


SHARE
Vol.16 • No. 1 • 2022
Back to Top