The Annals of Statistics
- Ann. Statist.
- Volume 39, Number 3 (2011), 1748-1775.
A majorization–minimization approach to variable selection using spike and slab priors
We develop a method to carry out MAP estimation for a class of Bayesian regression models in which coefficients are assigned with Gaussian-based spike and slab priors. The objective function in the corresponding optimization problem has a Lagrangian form in that regression coefficients are regularized by a mixture of squared l2 and l0 norms. A tight approximation to the l0 norm using majorization–minimization techniques is derived, and a coordinate descent algorithm in conjunction with a soft-thresholding scheme is used in searching for the optimizer of the approximate objective. Simulation studies show that the proposed method can lead to more accurate variable selection than other benchmark methods. Theoretical results show that under regular conditions, sign consistency can be established, even when the Irrepresentable Condition is violated. Results on posterior model consistency and estimation consistency, and an extension to parameter estimation in the generalized linear models are provided.
Ann. Statist., Volume 39, Number 3 (2011), 1748-1775.
First available in Project Euclid: 25 July 2011
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Yen, Tso-Jung. A majorization–minimization approach to variable selection using spike and slab priors. Ann. Statist. 39 (2011), no. 3, 1748--1775. doi:10.1214/11-AOS884. https://projecteuclid.org/euclid.aos/1311600282
- Supplementary material: Supplement File. In Supplementary Material, we provide brief discussions on the log-sum function, connections with other approaches, derivation of the soft-thresolding operator, and proofs of Theorems 5.1, 5.2 and 5.3.