Open Access
Translator Disclaimer
August 2021 On the rate of convergence of fully connected deep neural network regression estimates
Michael Kohler, Sophie Langer
Author Affiliations +
Ann. Statist. 49(4): 2231-2249 (August 2021). DOI: 10.1214/20-AOS2034

Abstract

Recent results in nonparametric regression show that deep learning, that is, neural network estimates with many hidden layers, are able to circumvent the so-called curse of dimensionality in case that suitable restrictions on the structure of the regression function hold. One key feature of the neural networks used in these results is that their network architecture has a further constraint, namely the network sparsity. In this paper, we show that we can get similar results also for least squares estimates based on simple fully connected neural networks with ReLU activation functions. Here, either the number of neurons per hidden layer is fixed and the number of hidden layers tends to infinity suitably fast for sample size tending to infinity, or the number of hidden layers is bounded by some logarithmic factor in the sample size and the number of neurons per hidden layer tends to infinity suitably fast for sample size tending to infinity. The proof is based on new approximation results concerning deep neural networks.

Citation

Download Citation

Michael Kohler. Sophie Langer. "On the rate of convergence of fully connected deep neural network regression estimates." Ann. Statist. 49 (4) 2231 - 2249, August 2021. https://doi.org/10.1214/20-AOS2034

Information

Received: 1 August 2020; Revised: 1 October 2020; Published: August 2021
First available in Project Euclid: 29 September 2021

Digital Object Identifier: 10.1214/20-AOS2034

Subjects:
Primary: 62G08
Secondary: 41A25 , 82C32

Keywords: curse of dimensionality , deep learning , neural networks , Nonparametric regression , rate of convergence

Rights: Copyright © 2021 Institute of Mathematical Statistics

JOURNAL ARTICLE
19 PAGES


SHARE
Vol.49 • No. 4 • August 2021
Back to Top