Bayesian Analysis

Bayesian Quantile Regression Based on the Empirical Likelihood with Spike and Slab Priors

Ruibin Xi, Yunxiao Li, and Yiming Hu

Full-text: Open access

Abstract

In this paper, we consider nonparametric Bayesian variable selection in quantile regression. The Bayesian model is based on the empirical likelihood, and the prior is chosen as the “spike-and-slab” prior–a mixture of a point mass at zero and a normal distribution. We show that the posterior distribution of the zero coefficients converges to a point mass at zero and that of the nonzero coefficients converges to a normal distribution. To further address the problem of low statistical efficiency in extreme quantile regression, we extend the Bayesian model such that it can integrate information at multiple quantiles to provide more accurate inference of extreme quantiles for homogenous error models. Simulation studies demonstrate that the proposed methods outperform or perform equally well compared with existing methods. We apply this Bayesian method to study the role of microRNAs on regulating gene expression and find that the regulation of microRNA may have a positive effect on the gene expression variation.

Article information

Source
Bayesian Anal., Volume 11, Number 3 (2016), 821-855.

Dates
First available in Project Euclid: 9 October 2015

Permanent link to this document
https://projecteuclid.org/euclid.ba/1444396540

Digital Object Identifier
doi:10.1214/15-BA975

Mathematical Reviews number (MathSciNet)
MR3543910

Zentralblatt MATH identifier
1357.62181

Keywords
model selection Gibbs sampler oracle property empirical process consistency

Citation

Xi, Ruibin; Li, Yunxiao; Hu, Yiming. Bayesian Quantile Regression Based on the Empirical Likelihood with Spike and Slab Priors. Bayesian Anal. 11 (2016), no. 3, 821--855. doi:10.1214/15-BA975. https://projecteuclid.org/euclid.ba/1444396540


Export citation

References

  • Berger, J. (2006). “The case for objective Bayesian analysis.” Bayesian Analysis, 1(3): 385–402.
  • Berger, J. O. and Strawderman, W. E. (1996). “Choice of hierarchical priors: admissibility in estimation of normal means.” The Annals of Statistics, 931–951.
  • Buchinsky, M. (1994). “Changes in the US wage structure 1963–1987: Application of quantile regression.” Econometrica: Journal of the Econometric Society, 405–458.
  • Chen, J., Variyath, A. M., and Abraham, B. (2008). “Adjusted empirical likelihood and its properties.” Journal of Computational and Graphical Statistics, 17(2): 426–443.
  • Chen, S. X. and Hall, P. (1993). “Smoothed empirical likelihood confidence intervals for quantiles.” The Annals of Statistics, 1166–1181.
  • Chen, S. X. and Van Keilegom, I. (2009). “A review on empirical likelihood methods for regression.” Test, 18(3): 415–447.
  • Chen, S. X. and Wong, C. (2009). “Smoothed block empirical likelihood for quantiles of weakly dependent processes.” Statistica Sinica, 19(1): 71.
  • Fan, J. and Li, R. (2001). “Variable selection via nonconcave penalized likelihood and its oracle properties.” Journal of the American Statistical Association, 96(456): 1348–1360.
  • Fenske, N., Kneib, T., and Hothorn, T. (2011). “Identifying risk factors for severe childhood malnutrition by boosting additive quantile regression.” Journal of the American Statistical Association, 106(494).
  • George, E. I. and McCulloch, R. E. (1993). “Variable selection via Gibbs sampling.” Journal of the American Statistical Association, 88(423): 881–889.
  • Hornstein, E. and Shomron, N. (2006). “Canalization of development by microRNAs.” Nature genetics, 38: S20–S24.
  • Hulmán, A., Witte, D. R., Kerényi, Z., Madarász, E., Tänczer, T., Bosnyák, Z., Szabó, E., Ferencz, V., Péterfalvi, A., and Tabák, A. G., et al. (2015). “Heterogeneous effect of gestational weight gain on birth weight: quantile regression analysis from a population-based screening.” Annals of Epidemiology 25(2):133–137.
  • Ishwaran, H. and Rao, J. S. (2005). “Spike and slab variable selection: frequentist and Bayesian strategies.” Annals of Statistics, 730–773.
  • Ishwaran, H. and Rao, J. S. (2011). “Consistency of spike and slab regression.” Statistics & Probability Letters, 81(12): 1920–1928.
  • Kim, M.-O. and Yang, Y. (2011). “Semiparametric approach to a random effects quantile regression model.” Journal of the American Statistical Association, 106(496).
  • Koenker, R. (2005). Quantile Regression. New York: Cambridge University Press.
  • Kottas, A. and Gelfand, A. E. (2001). “Bayesian semiparametric median regression modeling.” Journal of the American Statistical Association, 96(456): 1458–1468.
  • Lahiri, S. N., Mukhopadhyay, S., et al. (2012). “A penalized empirical likelihood method in high dimensions.” The Annals of Statistics, 40(5): 2511–2540.
  • Lazar, N. A. (2003). “Bayesian empirical likelihood.” Biometrika, 90(2): 319–326.
  • Lewis, B., Burge, C., and Bartel, D. (2005). “Conserved seed pairing, often flanked by adenosines, indicates that thousands of human genes are microRNA targets.” Cell, 120: 15–20.
  • Li, Q., Xi, R., and Lin, N. (2010). “Bayesian regularized quantile regression.” Bayesian Analysis, 5(3): 533–556.
  • Li, R., Zhong, W., and Zhu, L. (2012). “Feature screening via distance correlation learning.” Journal of the American Statistical Association, 107(499): 1129–1139.
  • Li, Y. and Zhu, J. (2008). “$L_{1}$-norm quantile regression.” Journal of Computational and Graphical Statistics, 17: 163–185.
  • Lin, N. and Chang, C. (2012). “Comment on article by Lum and Gelfand.” Bayesian Analysis, 7(2): 263–270.
  • Lu, J. and Clark, A. (2012). “Impact of microRNA regulation on variation in human gene expression.” Genome Research, 22: 1243–1254.
  • Machado, J. A. and Mata, J. (2005). “Counterfactual decomposition of changes in wage distributions using quantile regression.” Journal of applied Econometrics, 20(4): 445–465.
  • Mitchell, T. J. and Beauchamp, J. J. (1988). “Bayesian variable selection in linear regression.” Journal of the American Statistical Association, 83(404): 1023–1032.
  • Molanes Lopez, E., Keilegom, I., and Veraverbeke, N. (2009). “Empirical likelihood for non-smooth criterion functions.” Scandinavian Journal of Statistics, 36(3): 413–432.
  • Narisetty, N. N. and He, X. (2014). “Bayesian variable selection with shrinking and diffusing priors.” The Annals of Statistics, 42(2): 789–817.
  • Okada, K. and Samreth, S. (2012). “The effect of foreign aid on corruption: a quantile regression approach.” Economics Letters, 115(2): 240–243.
  • Owen, A. (1991). “Empirical likelihood for linear models.” The Annals of Statistics, 19(4): 1725–1747.
  • Owen, A. B. (1988). “Empirical likelihood ratio confidence intervals for a single functional.” Biometrika, 75(2): 237–249.
  • Owen, A. B. (2001). Empirical Likelihood. Chapman & Hall/CRC.
  • Qin, J. and Lawless, J. (1994). “Empirical likelihood and general estimating equations.” The Annals of Statistics, 300–325.
  • Roberts, G. O., Gelman, A., and Gilks, W. R. (1997). “Weak convergence and optimal scaling of random walk Metropolis algorithms.” The Annals of Applied Probability, 7(1): 110–120.
  • Roberts, G. O. and Rosenthal, J. S. (2001). “Optimal scaling for various Metropolis–Hastings algorithms.” Statistical Science, 16(4): 351–367.
  • Schennach, S. M. (2005). “Bayesian exponentially tilted empirical likelihood.” Biometrika, 92(1): 31–46.
  • Székely, G. J., Rizzo, M. L., Bakirov, N. K., et al. (2007). “Measuring and testing dependence by correlation of distances.” The Annals of Statistics, 35(6): 2769–2794.
  • Taddy, M. A. and Kottas, A. (2010). “A Bayesian nonparametric approach to inference for quantile regression.” Journal of Business & Economic Statistics, 28(3).
  • Tang, C. Y. and Leng, C. (2010). “Penalized high-dimensional empirical likelihood.” Biometrika, 97(4): 905–920.
  • Tibshirani, R. (1996). “Regression shrinkage and selection via the lasso.” Journal of the Royal Statistical Society, Series B (Statistical Methodology), 58: 267–288.
  • Tierney, L. (1994). “Markov chains for exploring posterior distributions.” the Annals of Statistics, 1701–1728.
  • Wu, C., Shen, Y., and Tang, T. (2009). “Evolution under canalization and the dual roles of microRNAs: a hypothesis.” Genome Research, 19: 734–743.
  • Xi, R., Li, Y., and Hu, Y. (2015). “Supplement to: Bayesian Quantile Regression Based on the Empirical Likelihood with spike and slab priors.” Bayesian Analysis.
  • Yang, Y. and He, X. (2012). “Bayesian empirical likelihood for quantile regression.” The Annals of Statistics, 40(2): 1102–1131.
  • Yu, K. and Moyeed, R. A. (2001). “Bayesian quantile regression.” Statistics & Probability Letters, 54(4): 437–447.
  • Zou, H. and Hastie, T. (2005). “Regularization and variable selection via the elastic net.” Journal of the Royal Statistical Society, Series B (Statistical Methodology), 67: 301–320.
  • Zou, H. and Yuan, M. (2008). “Regularized simultaneous model selection in multiple quantiles regression.” Computational Statistics & Data Analysis, 52(12): 5296–5304.

Supplemental materials