The Annals of Statistics
- Ann. Statist.
- Volume 33, Number 2 (2005), 730-773.
Spike and slab variable selection: Frequentist and Bayesian strategies
Hemant Ishwaran and J. Sunil Rao
Abstract
Variable selection in the linear regression model takes many apparent faces from both frequentist and Bayesian standpoints. In this paper we introduce a variable selection method referred to as a rescaled spike and slab model. We study the importance of prior hierarchical specifications and draw connections to frequentist generalized ridge regression estimation. Specifically, we study the usefulness of continuous bimodal priors to model hypervariance parameters, and the effect scaling has on the posterior mean through its relationship to penalization. Several model selection strategies, some frequentist and some Bayesian in nature, are developed and studied theoretically. We demonstrate the importance of selective shrinkage for effective variable selection in terms of risk misclassification, and show this is achieved using the posterior from a rescaled spike and slab model. We also show how to verify a procedure’s ability to reduce model uncertainty in finite samples using a specialized forward selection strategy. Using this tool, we illustrate the effectiveness of rescaled spike and slab models in reducing model uncertainty.
Article information
Source
Ann. Statist., Volume 33, Number 2 (2005), 730-773.
Dates
First available in Project Euclid: 26 May 2005
Permanent link to this document
https://projecteuclid.org/euclid.aos/1117114335
Digital Object Identifier
doi:10.1214/009053604000001147
Mathematical Reviews number (MathSciNet)
MR2163158
Zentralblatt MATH identifier
1068.62079
Subjects
Primary: 62J07: Ridge regression; shrinkage estimators
Secondary: 62J05: Linear regression
Keywords
Generalized ridge regression hypervariance model averaging model uncertainty ordinary least squares penalization rescaling shrinkage stochastic variable selection Zcut
Citation
Ishwaran, Hemant; Rao, J. Sunil. Spike and slab variable selection: Frequentist and Bayesian strategies. Ann. Statist. 33 (2005), no. 2, 730--773. doi:10.1214/009053604000001147. https://projecteuclid.org/euclid.aos/1117114335