I would like to congratulate Johannes Schmidt–Hieber on a very interesting paper in which he considers regression functions belonging to the class of so-called compositional functions and analyzes the ability of estimators based on the multivariate nonparametric regression model of deep neural networks to achieve minimax rates of convergence.
In my discussion, I will first regard such a type of result from the general viewpoint of the theoretical foundations of deep neural networks. This will be followed by a discussion from the viewpoint of expressivity, optimization and generalization. Finally, I will consider some specific aspects of the main result.
"Discussion of: “Nonparametric regression using deep neural networks with ReLU activation function”." Ann. Statist. 48 (4) 1902 - 1905, August 2020. https://doi.org/10.1214/19-AOS1911