Bayesian Analysis

Penalized regression, standard errors, and Bayesian lassos

George Casella, Malay Ghosh, Jeff Gill, and Minjung Kyung

Full-text: Open access

Abstract

Penalized regression methods for simultaneous variable selection and coefficient estimation, especially those based on the lasso of Tibshirani (1996), have received a great deal of attention in recent years, mostly through frequentist models. Properties such as consistency have been studied, and are achieved by different lasso variations. Here we look at a fully Bayesian formulation of the problem, which is flexible enough to encompass most versions of the lasso that have been previously considered. The advantages of the hierarchical Bayesian formulations are many. In addition to the usual ease-of-interpretation of hierarchical models, the Bayesian formulation produces valid standard errors (which can be problematic for the frequentist lasso), and is based on a geometrically ergodic Markov chain. We compare the performance of the Bayesian lassos to their frequentist counterparts using simulations, data sets that previous lasso papers have used, and a difficult modeling problem for predicting the collapse of governments around the world. In terms of prediction mean squared error, the Bayesian lasso performance is similar to and, in some cases, better than, the frequentist lasso.

Article information

Source
Bayesian Anal. Volume 5, Number 2 (2010), 369-411.

Dates
First available in Project Euclid: 20 June 2012

Permanent link to this document
https://projecteuclid.org/euclid.ba/1340218343

Digital Object Identifier
doi:10.1214/10-BA607

Mathematical Reviews number (MathSciNet)
MR2719657

Zentralblatt MATH identifier
1330.62289

Keywords
Hierarchical Models Gibbs Sampling Geometric Ergodicity Variable Selection

Citation

Kyung, Minjung; Gill, Jeff; Ghosh, Malay; Casella, George. Penalized regression, standard errors, and Bayesian lassos. Bayesian Anal. 5 (2010), no. 2, 369--411. doi:10.1214/10-BA607. https://projecteuclid.org/euclid.ba/1340218343.


Export citation