The Annals of Statistics

Bayesian empirical likelihood for quantile regression

Yunwen Yang and Xuming He

Full-text: Open access

Abstract

Bayesian inference provides a flexible way of combining data with prior information. However, quantile regression is not equipped with a parametric likelihood, and therefore, Bayesian inference for quantile regression demands careful investigation. This paper considers the Bayesian empirical likelihood approach to quantile regression. Taking the empirical likelihood into a Bayesian framework, we show that the resultant posterior from any fixed prior is asymptotically normal; its mean shrinks toward the true parameter values, and its variance approaches that of the maximum empirical likelihood estimator. A more interesting case can be made for the Bayesian empirical likelihood when informative priors are used to explore commonality across quantiles. Regression quantiles that are computed separately at each percentile level tend to be highly variable in the data sparse areas (e.g., high or low percentile levels). Through empirical likelihood, the proposed method enables us to explore various forms of commonality across quantiles for efficiency gains. By using an MCMC algorithm in the computation, we avoid the daunting task of directly maximizing empirical likelihood. The finite sample performance of the proposed method is investigated empirically, where substantial efficiency gains are demonstrated with informative priors on common features across several percentile levels. A theoretical framework of shrinking priors is used in the paper to better understand the power of the proposed method.

Article information

Source
Ann. Statist. Volume 40, Number 2 (2012), 1102-1131.

Dates
First available in Project Euclid: 18 July 2012

Permanent link to this document
https://projecteuclid.org/euclid.aos/1342625463

Digital Object Identifier
doi:10.1214/12-AOS1005

Mathematical Reviews number (MathSciNet)
MR2985945

Zentralblatt MATH identifier
1274.62458

Subjects
Primary: 62J05: Linear regression 62F12: Asymptotic properties of estimators
Secondary: 62G20: Asymptotic properties

Keywords
Efficiency empirical likelihood high quantiles prior posterior

Citation

Yang, Yunwen; He, Xuming. Bayesian empirical likelihood for quantile regression. Ann. Statist. 40 (2012), no. 2, 1102--1131. doi:10.1214/12-AOS1005. https://projecteuclid.org/euclid.aos/1342625463.


Export citation

References

  • Chang, I. H. and Mukerjee, R. (2008). Bayesian and frequentist confidence intervals arising from empirical-type likelihoods. Biometrika 95 139–147.
  • Chen, S. X. and Hall, P. (1993). Smoothed empirical likelihood confidence intervals for quantiles. Ann. Statist. 21 1166–1181.
  • Chen, J., Sitter, R. R. and Wu, C. (2002). Using empirical likelihood methods to obtain range restricted weights in regression estimators for surveys. Biometrika 89 230–237.
  • Chen, S. X. and Van Keilegom, I. (2009). A review on empirical likelihood methods for regression. TEST 18 415–447.
  • Chen, S. X. and Wong, C. M. (2009). Smoothed block empirical likelihood for quantiles of weakly dependent processes. Statist. Sinica 19 71–81.
  • Chen, K., Ying, Z., Zhang, H. and Zhao, L. (2008). Analysis of least absolute deviation. Biometrika 95 107–122.
  • Chernozhukov, V. and Hong, H. (2003). An MCMC approach to classical estimation. J. Econometrics 115 293–346.
  • Chung, Y. and Dunson, D. B. (2009). Nonparametric Bayes conditional distribution modeling with variable selection. J. Amer. Statist. Assoc. 104 1646–1660.
  • Dunson, D. B., Pillai, N. and Park, J.-H. (2007). Bayesian density regression. J. R. Stat. Soc. Ser. B Stat. Methodol. 69 163–183.
  • Dunson, D. B. and Taylor, J. A. (2005). Approximate Bayesian inference for quantiles. J. Nonparametr. Stat. 17 385–400.
  • Fang, K.-T. and Mukerjee, R. (2006). Empirical-type likelihoods allowing posterior credible sets with frequentist validity: Higher-order asymptotics. Biometrika 93 723–733.
  • Geraci, M. and Bottai, M. (2007). Quantile regression for longitudinal data using the asymmetric Laplace distribution. Biostatistics 8 140–154.
  • Gilks, W. R., Richardson, S. and Spiegelhalter, D. J., eds. (1996). Markov Chain Monte Carlo in Practice. Chapman & Hall, London.
  • Gutenbrunner, C. and Jurečková, J. (1992). Regression rank scores and regression quantiles. Ann. Statist. 20 305–330.
  • Hahn, J. (1997). Bayesian bootstrap of the quantile regression estimator: A large sample study. Internat. Econom. Rev. 38 795–808.
  • Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57 97–109.
  • He, X. and Shao, Q.-M. (1996). A general Bahadur representation of $M$-estimators and its application to linear regression with nonstochastic designs. Ann. Statist. 24 2608–2630.
  • He, X., Zhu, Z.-Y. and Fung, W.-K. (2002). Estimation in a semiparametric model for longitudinal data with unspecified dependence structure. Biometrika 89 579–590.
  • Horowitz, J. L. (1998). Bootstrap methods for median regression models. Econometrica 66 1327–1351.
  • Kim, M.-O. and Yang, Y. (2011). Semiparametric approach to a random effects quantile regression model. J. Amer. Statist. Assoc. 106 1405–1417.
  • Kocherginsky, M., He, X. and Mu, Y. (2005). Practical confidence intervals for regression quantiles. J. Comput. Graph. Statist. 14 41–55.
  • Koenker, R. (2005). Quantile Regression. Econometric Society Monographs 38. Cambridge Univ. Press, Cambridge.
  • Koenker, R. and Bassett, G. Jr. (1978). Regression quantiles. Econometrica 46 33–50.
  • Koenker, R. and Bassett, G. Jr. (1982). Robust tests for heteroscedasticity based on regression quantiles. Econometrica 50 43–61.
  • Kosorok, M. R. (2008). Introduction to Empirical Processes and Semiparametric Inference. Springer, New York.
  • Kottas, A. and Gelfand, A. E. (2001). Bayesian semiparametric mdeian regression modeling. J. Amer. Statist. Assoc. 96 1458–1468.
  • Kottas, A. and Krnjajić, M. (2009). Bayesian semiparametric modelling in quantile regression. Scand. J. Stat. 36 297–319.
  • Lancaster, T. and Jun, S. J. (2010). Bayesian quantile regression methods. J. Appl. Econometrics 25 287–307.
  • Lazar, N. A. (2003). Bayesian empirical likelihood. Biometrika 90 319–326.
  • Molanes Lopez, E. M., Van Keilegom, I. and Veraverbeke, N. (2009). Empirical likelihood for non-smooth criterion functions. Scand. J. Stat. 36 413–432.
  • Monahan, J. F. and Boos, D. D. (1992). Proper likelihoods for Bayesian analysis. Biometrika 79 271–278.
  • Müller, P., Erkanli, A. and West, M. (1996). Bayesian curve fitting using multivariate normal mixtures. Biometrika 83 67–79.
  • Müller, P. and Quintana, F. A. (2004). Nonparametric Bayesian data analysis. Statist. Sci. 19 95–110.
  • Newey, W. K. and Smith, R. J. (2004). Higher order properties of GMM and generalized empirical likelihood estimators. Econometrica 72 219–255.
  • Norets, A. and Pelenis, J. (2010). Posterior consistency in conditional distribution estimation by covariate dependent mixtures. Unpublished manuscript.
  • Otsu, T. (2008). Conditional empirical likelihood estimation and inference for quantile regression models. J. Econometrics 142 508–538.
  • Owen, A. B. (1988). Empirical likelihood ratio confidence intervals for a single functional. Biometrika 75 237–249.
  • Owen, A. B. (2001). Empirical Likelihood. Chapman & Hall/CRC, Boca Raton, FL.
  • Pati, D., Dunson, D. and Tokdary, S. (2010). Posterior consistency in conditional distribution estimation. Unpublished manuscript.
  • Qin, J. and Lawless, J. (1994). Empirical likelihood and general estimating equations. Ann. Statist. 22 300–325.
  • Reich, B. J., Bondell, H. D. and Wang, H. J. (2008). Flexible Bayesian quantile regression for independent and clustered data. Biostatistics 11 337–352.
  • Reich, B. J., Fuentes, M. and Dunson, D. B. (2011). Bayesian spatial quantile regression. J. Amer. Statist. Assoc. 106 6–20.
  • Ren, J.-J. (2008). Smoothed weighted empirical likelihood ratio confidence intervals for quantiles. Bernoulli 14 725–748.
  • Schennach, S. M. (2005). Bayesian exponentially tilted empirical likelihood. Biometrika 92 31–46.
  • Schennach, S. M. (2007). Point estimation with exponentially tilted empirical likelihood. Ann. Statist. 35 634–672.
  • Uppala, S. M., K Ållberg, P. W., Simmons, A. J., Andrae, U., Da Costa Bechtold, V., Fiorino, M., Gibson, J. K., Haseler, J., Hernandez, A., Kelly, G. A., Li, X., Onogi, K., Saarinen, S., Sokka, N., Allan, R. P., Andersson, E., Arpe, K., Balmaseda, M. A., Beljaars, A. C. M., Berg, L. V. D., Bidlot, J., Bormann, N., Caires, S., Chevallier, F., Dethof, A., Dragosavac, M., Fisher, M., Fuentes, M., Hagemann, S., Hólm, E., Hoskins, B. J., Isaksen, L., Janssen, P. A. E. M., Jenne, R., McNally, A. P., Mahfouf, J. F., Morcrette, J. J., Rayner, N. A., Saunders, R. W., Simon, P., Sterl, A., Trenberth, K. E., Untch, A., Vasiljevic, D., Viterbo, P. and Woollen, J. (2005). The ERA-40 re-analysis. Quarterly Journal of the Royal Meteorological Society 131 2961–3012.
  • van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge.
  • Wang, H. J. and Zhu, Z. (2011). Empirical likelihood for quantile regression models with longitudinal data. J. Statist. Plann. Inference 141 1603–1615.
  • Whang, Y.-J. (2006). Smoothed empirical likelihood methods for quantile regression models. Econometric Theory 22 173–205.
  • Wilby, R. L. and Wigley, T. M. L. (1997). Downscaling general circulation model output: A review of methods and limitations. Progress in Physical Geography 21 530–548.
  • Yang, Y. and He, X. (2012). Supplement to “Bayesian empirical likelihood for quantile regression.” DOI:10.1214/12-AOS1005SUPP.
  • Yin, G. (2009). Bayesian generalized method of moments. Bayesian Anal. 4 191–207.
  • Yu, K. and Moyeed, R. A. (2001). Bayesian quantile regression. Statist. Probab. Lett. 54 437–447.
  • Yue, Y. R. and Rue, H. (2011). Bayesian inference for additive mixed quantile regression models. Comput. Statist. Data Anal. 55 84–96.
  • Zou, H. and Yuan, M. (2008). Composite quantile regression and the oracle model selection theory. Ann. Statist. 36 1108–1126.

Supplemental materials

  • Supplementary material: Supplement to “Bayesian empirical likelihood for quantile regression”. The supplementary material contains additional details on the implementation of the Bayesian computations used in the empirical studies reported in this paper.