Bayesian Analysis

Bayesian regularized quantile regression

Qing Li, Nan Lin, and Ruibin Xi

Full-text: Open access

Abstract

Regularization, e.g. lasso, has been shown to be effective in quantile regression in improving the prediction accuracy (Li and Zhu, 2008; Wu and Liu, 2009). This paper studies regularization in quantile regressions from a Bayesian perspective. By proposing a hierarchical model framework, we give a generic treatment to a set of regularization approaches, including lasso, group lasso and elastic net penalties. Gibbs samplers are derived for all cases. This is the first work to discuss regularized quantile regression with the group lasso penalty and the elastic net penalty. Both simulated and real data examples show that Bayesian regularized quantile regression methods often outperform quantile regression without regularization and their non-Bayesian counterparts with regularization.

Article information

Source
Bayesian Anal. Volume 5, Number 3 (2010), 533-556.

Dates
First available in Project Euclid: 22 June 2012

Permanent link to this document
https://projecteuclid.org/euclid.ba/1340380540

Digital Object Identifier
doi:10.1214/10-BA521

Mathematical Reviews number (MathSciNet)
MR2719666

Zentralblatt MATH identifier
1330.62143

Keywords
Quantile regression Regularization Gibbs sampler Bayesian analysis Lasso Elastic net Group lasso

Citation

Li, Qing; Xi, Ruibin; Lin, Nan. Bayesian regularized quantile regression. Bayesian Anal. 5 (2010), no. 3, 533--556. doi:10.1214/10-BA521. https://projecteuclid.org/euclid.ba/1340380540


Export citation

References

  • Andrews, D. F. and Mallows, C. L. (1974). "Scale Mixtures of Normal Distributions." Journal of the Royal Statistical Society, Ser. B, 36: 99–102.
  • Bae, K. and Mallick, B. (2004). "Gene Selection Using a Two-Level Hierarchical Bayesian" Model. Bioinformatics, 20: 3423–3430.
  • Bakin, S. (1999). "Adaptive Regression and Model Selection in Data Mining Problems." PhD thesis, Australian National University, Canberra.
  • Fan, J. and Li, R. (2001). "Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties." Journal of the American Statistical Association, 96: 1348–1360.
  • Geraci, M. and Bottai, M. (2007). "Quantile Regression for Longitudinal Data Using the Asymmetric Laplace" Distribution. Biostatistics, 8: 140–154.
  • Hanson, T. and Johnson, W. (2002). "Modeling Regression Error with a Mixture of Pólya trees." Journal of the American Statistical Association, 97: 1020–1033.
  • Harrison, D. and Rubinfeld, D. L. (1978). "Hedonic Prices and the Demand for Clean Air." Journal of Environmental Economics and Management, 5: 81–102.
  • Hjort, N. and Walker, S. (2009). "Quantile Pyramids for Bayesian" Nonparametrics. Annals of Statistics, 37: 105–131.
  • Jørgensen, B. (1982). Statistical Properties of the Generalized Inverse Gaussian Distribution, Lecture Notes in Statistics, volume 9. Springer-Verlag.
  • Koenker, R. (2004). "Quantile Regression for Longitudinal Data." Journal of Multivariate Analysis, 91: 74–89.
  • –- (2005). Quantile Regression. New York: Cambridge University Press.
  • Koenker, R. and Bassett, G. W. (1978). "Regression Quantiles." Economerrica, 46: 33–50.
  • Kottas, A. and Gelfand, A. (2001). "Bayesian" Semiparametric Median Regression Modeling." Journal of the American Statistical Association, 96: 1458–1468.
  • Kottas, A. and Krnjajić, M. (2009). "Bayesian" Semiparametric Modeling in Quantile Regression." Scandinavian Journal of Statistics, 36: 297–319.
  • Kozumi, H. and Kobayashi, G. (2009). "Gibbs Sampling Methods for Bayesian Quantile Regression." Technical report, Graduate School of Business Administration, Kobe University. http://www.b.kobe-u.ac.jp/paper/2009_02.pdf
  • Li, Y. and Zhu, J. (2008). "$L_1$-Norm Quantile Regression." Journal of Computational and Graphical Statistics, 17: 163–185.
  • Meier, L., van de Geer, S., and B$\mbox{\"{u}}$hlmann, P. (2008). "The Group Lasso for Logistic Regression." Journal of the Royal Statistical Society, Ser. B, 70(1): 53–71.
  • Park, T. and Casella, G. (2008). "The Bayesian" Lasso. Journal of the American Statistical Association, 103: 681–686.
  • R Development Core Team (2005). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org
  • Reed, C., Dunson, D., and Yu, K. (2009). "Bayesian" variable selection in quantile regression." Technical report, Department of Mathematical Sciences, Brunel University.
  • Reed, C. and Yu, K. (2009). "An efficient Gibbs" sampler for Bayesian quantile regression. Technical report, Department of Mathematical Sciences, Brunel University.
  • Reich, B., Bondell, H., and Wang, H. (2009). "Flexible Bayesian" Quantile Regression for Independent and Clustered Data. Biostatistics. Doi:10.1093/biostatistics/kxp049.
  • Tibshirani, R. (1996). "Regression Shrinkage and Selection via the Lasso." Journal of the Royal Statistical Society, Ser. B, 58: 267–288.
  • Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., and Knight, K. (2005). "Sparsity and Smoothness via the Fused Lasso." Journal of the Royal Statistical Society: Ser. B, 67: 91–108.
  • Tsionas, E. (2003). "Bayesian" Quantile Inference." Journal of Statistical Computation and Simulation, 73: 659–674.
  • Walker, S. and Mallick, B. (1999). "A Bayesian" Semiparametric Accelerated Failure Time Model. Biometrics, 55: 477–483.
  • Wang, H., Li, G., and Jiang, G. (2007). "Robust Regression Shrinkage and Consistent Variable Selection Through the LAD-Lasso." Journal of Business & Economic Statistics, 25: 347–355.
  • Wu, Y. and Liu, Y. (2009). "Variable Selection in Quantile Regression." Statistica Sinica, 19: 801–817.
  • Yu, K. and Moyeed, R. (2001). "Bayesian" Quantile Regression." Statistics & Probability Letters, 54: 437–447.
  • Yuan, M. and Lin, Y. (2006). "Model Selection and Estimation in Regression with Grouped Variables." Journal of the Royal Statistical Society, Ser. B, 68: 49–67.
  • Zou, H. and Hastie, T. (2005). "Regularization and Variable Selection via the Elastic Net." Journal of the Royal Statistical Society, Ser. B, 67: 301–320.
  • Zou, H. and Yuan, M. (2008). "Composite Quantile Regression and The Oracle Model Selection Theory." Annals of Statistics, 36: 1108–1126.
  • –- (2008). "Regularized Simultaneous Model Selection in Multiple Quantiles Regression." Computational Statistics & Data Analysis, 52: 5296–5304.