Translator Disclaimer
May 2021 A Selective Overview of Deep Learning
Jianqing Fan, Cong Ma, Yiqiao Zhong
Author Affiliations +
Statist. Sci. 36(2): 264-290 (May 2021). DOI: 10.1214/20-STS783


Deep learning has achieved tremendous success in recent years. In simple words, deep learning uses the composition of many nonlinear functions to model the complex dependency between input features and labels. While neural networks have a long history, recent advances have significantly improved their empirical performance in computer vision, natural language processing and other predictive tasks. From the statistical and scientific perspective, it is natural to ask: What is deep learning? What are the new characteristics of deep learning, compared with classical statistical methods? What are the theoretical foundations of deep learning?

To answer these questions, we introduce common neural network models (e.g., convolutional neural nets, recurrent neural nets, generative adversarial nets) and training techniques (e.g., stochastic gradient descent, dropout, batch normalization) from a statistical point of view. Along the way, we highlight new characteristics of deep learning (including depth and overparametrization) and explain their practical and theoretical benefits. We also sample recent results on theories of deep learning, many of which are only suggestive. While a complete understanding of deep learning remains elusive, we hope that our perspectives and discussions serve as a stimulus for new statistical research.


Download Citation

Jianqing Fan. Cong Ma. Yiqiao Zhong. "A Selective Overview of Deep Learning." Statist. Sci. 36 (2) 264 - 290, May 2021.


Published: May 2021
First available in Project Euclid: 19 April 2021

Digital Object Identifier: 10.1214/20-STS783

Keywords: approximation theory , Generalization error , neural networks , overparametrization , Stochastic gradient descent

Rights: Copyright © 2021 Institute of Mathematical Statistics


This article is only available to subscribers.
It is not available for individual sale.

Vol.36 • No. 2 • May 2021
Back to Top