- Bayesian Anal.
- Volume 10, Number 1 (2015), 75-107.
Power-Expected-Posterior Priors for Variable Selection in Gaussian Linear Models
In the context of the expected-posterior prior (EPP) approach to Bayesian variable selection in linear models, we combine ideas from power-prior and unit-information-prior methodologies to simultaneously (a) produce a minimally-informative prior and (b) diminish the effect of training samples. The result is that in practice our power-expected-posterior (PEP) methodology is sufficiently insensitive to the size of the training sample, due to PEP’s unit-information construction, that one may take equal to the full-data sample size and dispense with training samples altogether. This promotes stability of the resulting Bayes factors, removes the arbitrariness arising from individual training-sample selections, and greatly increases computational speed, allowing many more models to be compared within a fixed CPU budget. We find that, under an independence Jeffreys (reference) baseline prior, the asymptotics of PEP Bayes factors are equivalent to those of Schwartz’s Bayesian Information Criterion (BIC), ensuring consistency of the PEP approach to model selection. Our PEP prior, due to its unit-information structure, leads to a variable-selection procedure that — in our empirical studies — (1) is systematically more parsimonious than the basic EPP with minimal training sample, while sacrificing no desirable performance characteristics to achieve this parsimony; (2) is robust to the size of the training sample, thus enjoying the advantages described above arising from the avoidance of training samples altogether; and (3) identifies maximum-a-posteriori models that achieve better out-of-sample predictive performance than that provided by standard EPPs, the -prior, the hyper- prior, non-local priors, the Least Absolute Shrinkage and Selection Operator (LASSO) and Smoothly-Clipped Absolute Deviation (SCAD) methods.
Bayesian Anal. Volume 10, Number 1 (2015), 75-107.
First available in Project Euclid: 28 January 2015
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Bayesian variable selection Bayes factors Consistency Expected-posterior priors Gaussian linear models g-prior Hyper-g prior LASSO Non-local priors Prior compatibility Power-prior Training samples SCAD Unit-information prior
Fouskakis, Dimitris; Ntzoufras, Ioannis; Draper, David. Power-Expected-Posterior Priors for Variable Selection in Gaussian Linear Models. Bayesian Anal. 10 (2015), no. 1, 75--107. doi:10.1214/14-BA887. https://projecteuclid.org/euclid.ba/1422468424
- Supplementary material: Web Appendix to “Power-Expected-Posterior Priors for Variable Selection in Gaussian Linear Models”. The Appendix is available in a web supplement at bayesian.org.