Open Access
December 2022 Bayesian Hierarchical Stacking: Some Models Are (Somewhere) Useful
Yuling Yao, Gregor Pirš, Aki Vehtari, Andrew Gelman
Author Affiliations +
Bayesian Anal. 17(4): 1043-1071 (December 2022). DOI: 10.1214/21-BA1287


Stacking is a widely used model averaging technique that asymptotically yields optimal predictions among linear averages. We show that stacking is most effective when model predictive performance is heterogeneous in inputs, and we can further improve the stacked mixture with a hierarchical model. We generalize stacking to Bayesian hierarchical stacking. The model weights are varying as a function of data, partially-pooled, and inferred using Bayesian inference. We further incorporate discrete and continuous inputs, other structured priors, and time series and longitudinal data. To verify the performance gain of the proposed method, we derive theory bounds, and demonstrate on several applied problems.

Funding Statement

The authors thank the National Science Foundation, Institute of Education Sciences, Office of Naval Research, National Institutes of Health, Sloan Foundation, Schmidt Futures, and the Academy of Finland Flagship programme: Finnish Center for Artificial Intelligence (FCAI) for partial financial support. Gregor Pirš is supported by the Slovenian Research Agency young researcher grant.


Download Citation

Yuling Yao. Gregor Pirš. Aki Vehtari. Andrew Gelman. "Bayesian Hierarchical Stacking: Some Models Are (Somewhere) Useful." Bayesian Anal. 17 (4) 1043 - 1071, December 2022.


Published: December 2022
First available in Project Euclid: 27 September 2021

MathSciNet: MR4506021
Digital Object Identifier: 10.1214/21-BA1287

Keywords: Bayesian hierarchical modeling , conditional prediction , covariate shift , model averaging , prior construction , stacking

Vol.17 • No. 4 • December 2022
Back to Top