June 2021 Learning models with uniform performance via distributionally robust optimization
John C. Duchi, Hongseok Namkoong
Author Affiliations +
Ann. Statist. 49(3): 1378-1406 (June 2021). DOI: 10.1214/20-AOS2004


A common goal in statistics and machine learning is to learn models that can perform well against distributional shifts, such as latent heterogeneous subpopulations, unknown covariate shifts or unmodeled temporal effects. We develop and analyze a distributionally robust stochastic optimization (DRO) framework that learns a model providing good performance against perturbations to the data-generating distribution. We give a convex formulation for the problem, providing several convergence guarantees. We prove finite-sample minimax upper and lower bounds, showing that distributional robustness sometimes comes at a cost in convergence rates. We give limit theorems for the learned parameters, where we fully specify the limiting distribution so that confidence intervals can be computed. On real tasks including generalizing to unknown subpopulations, fine-grained recognition and providing good tail performance, the distributionally robust approach often exhibits improved performance.

Funding Statement

Both authors were supported by the SAIL-Toyota Center for AI Research. J. C. Duchi was supported by National Science Foundation Award NSF-CAREER-1553086 and Office of Naval Research YIP Award N00014-19-2288. H. Namkoong was supported by Samsung Fellowship.


Download Citation

John C. Duchi. Hongseok Namkoong. "Learning models with uniform performance via distributionally robust optimization." Ann. Statist. 49 (3) 1378 - 1406, June 2021. https://doi.org/10.1214/20-AOS2004


Received: 1 October 2018; Revised: 1 July 2020; Published: June 2021
First available in Project Euclid: 9 August 2021

MathSciNet: MR4298868
zbMATH: 1473.62019
Digital Object Identifier: 10.1214/20-AOS2004

Primary: 62C20 , 62F12 , 68Q32

Keywords: Minimax optimality , risk-averse learning , Robust optimization

Rights: Copyright © 2021 Institute of Mathematical Statistics


This article is only available to subscribers.
It is not available for individual sale.

Vol.49 • No. 3 • June 2021
Back to Top