Annals of Statistics
- Ann. Statist.
- Volume 40, Number 3 (2012), 1550-1577.
Criteria for Bayesian model choice with application to variable selection
M. J. Bayarri, J. O. Berger, A. Forte, and G. García-Donato
Full-text: Open access
Abstract
In objective Bayesian model selection, no single criterion has emerged as dominant in defining objective prior distributions. Indeed, many criteria have been separately proposed and utilized to propose differing prior choices. We first formalize the most general and compelling of the various criteria that have been suggested, together with a new criterion. We then illustrate the potential of these criteria in determining objective model selection priors by considering their application to the problem of variable selection in normal linear models. This results in a new model selection objective prior with a number of compelling properties.
Article information
Source
Ann. Statist., Volume 40, Number 3 (2012), 1550-1577.
Dates
First available in Project Euclid: 5 September 2012
Permanent link to this document
https://projecteuclid.org/euclid.aos/1346850065
Digital Object Identifier
doi:10.1214/12-AOS1013
Mathematical Reviews number (MathSciNet)
MR3015035
Zentralblatt MATH identifier
1257.62023
Subjects
Primary: 62J05: Linear regression 62J15: Paired and multiple comparisons
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures
Keywords
Model selection variable selection objective Bayes
Citation
Bayarri, M. J.; Berger, J. O.; Forte, A.; García-Donato, G. Criteria for Bayesian model choice with application to variable selection. Ann. Statist. 40 (2012), no. 3, 1550--1577. doi:10.1214/12-AOS1013. https://projecteuclid.org/euclid.aos/1346850065
References
- Abramowitz, M. and Stegun, I. A. (1964). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Dover, New York.
- Bayarri, M. J. and García-Donato, G. (2008). Generalization of Jeffreys divergence-based priors for Bayesian hypothesis testing. J. R. Stat. Soc. Ser. B Stat. Methodol. 70 981–1003.Mathematical Reviews (MathSciNet): MR2530326
Digital Object Identifier: doi:10.1111/j.1467-9868.2008.00667.x - Berger, J. (1980). A robust generalized Bayes estimator and confidence region for a multivariate normal mean. Ann. Statist. 8 716–761.Mathematical Reviews (MathSciNet): MR572619
Zentralblatt MATH: 0464.62026
Digital Object Identifier: doi:10.1214/aos/1176345068
Project Euclid: euclid.aos/1176345068 - Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer, New York.
- Berger, J. O., Bayarri, M. J. and Pericchi, L. R. (2012). The effective sample size. Econometric Reviews. To appear.
- Berger, J. O., Ghosh, J. K. and Mukhopadhyay, N. (2003). Approximations and consistency of Bayes factors as model dimension grows. J. Statist. Plann. Inference 112 241–258.Mathematical Reviews (MathSciNet): MR1961733
Zentralblatt MATH: 1026.62018
Digital Object Identifier: doi:10.1016/S0378-3758(02)00336-1 - Berger, J. O. and Pericchi, L. R. (1996). The intrinsic Bayes factor for model selection and prediction. J. Amer. Statist. Assoc. 91 109–122.Mathematical Reviews (MathSciNet): MR1394065
Zentralblatt MATH: 0870.62021
Digital Object Identifier: doi:10.1080/01621459.1996.10476668 - Berger, J. O., Pericchi, L. R. and Varshavsky, J. A. (1998). Bayes factors and marginal distributions in invariant situations. Sankhyā Ser. A 60 307–321.Mathematical Reviews (MathSciNet): MR1718789
- Berger, J. O. and Pericchi, L. R. (2001). Objective Bayesian methods for model selection: Introduction and comparison. In Model Selection. Institute of Mathematical Statistics Lecture Notes—Monograph Series 38 135–207. IMS, Beachwood, OH.
- Casella, G., Girón, F. J., Martínez, M. L. and Moreno, E. (2009). Consistency of Bayesian procedures for variable selection. Ann. Statist. 37 1207–1228.Mathematical Reviews (MathSciNet): MR2509072
Zentralblatt MATH: 1160.62004
Digital Object Identifier: doi:10.1214/08-AOS606
Project Euclid: euclid.aos/1239369020 - Cui, W. and George, E. I. (2008). Empirical Bayes vs. fully Bayes variable selection. J. Statist. Plann. Inference 138 888–900.Mathematical Reviews (MathSciNet): MR2416869
Zentralblatt MATH: 1130.62007
Digital Object Identifier: doi:10.1016/j.jspi.2007.02.011 - De Santis, F. and Spezzaferri, F. (1999). Methods for default and roubst Bayesian model comparison: The fractional Bayes factor approach. International Statistical Review 67 267–286.
- Fernández, C., Ley, E. and Steel, M. F. J. (2001). Benchmark priors for Bayesian model averaging. J. Econometrics 100 381–427.Mathematical Reviews (MathSciNet): MR1820410
Zentralblatt MATH: 1091.62507
Digital Object Identifier: doi:10.1016/S0304-4076(00)00076-2 - Forte, A. (2011). Objective Bayesian criteria for variable selection. Ph.D. thesis, Univ. de Valencia.
- Ghosh, J. K. and Samanta, T. (2002). Nonsubjective Bayes testing—an overview. J. Statist. Plann. Inference 103 205–223.Mathematical Reviews (MathSciNet): MR1896993
Zentralblatt MATH: 0989.62017
Digital Object Identifier: doi:10.1016/S0378-3758(01)00222-1 - Guo, R. and Speckman, P. L. (2009). Bayes factors consistency in linear models. Presented in O’Bayes 09 conference.
- Hsiao, C. K. (1997). Approximate Bayes factors when a mode occurs on the boundary. J. Amer. Statist. Assoc. 92 656–663.Mathematical Reviews (MathSciNet): MR1467856
Zentralblatt MATH: 1067.62519
Digital Object Identifier: doi:10.1080/01621459.1997.10474017 - Jeffreys, H. (1961). Theory of Probability, 3rd ed. Clarendon, Oxford.Mathematical Reviews (MathSciNet): MR187257
- Kass, R. E. and Raftery, A. E. (1995). Bayes factors. J. Amer. Statist. Assoc. 90 773–795.
- Kass, R. E. and Vaidyanathan, S. K. (1992). Approximate Bayes factors and orthogonal parameters, with application to testing equality of two binomial proportions. J. Roy. Statist. Soc. Ser. B 54 129–144.Mathematical Reviews (MathSciNet): MR1157716
- Kass, R. E. and Wasserman, L. (1995). A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. J. Amer. Statist. Assoc. 90 928–934.Mathematical Reviews (MathSciNet): MR1354008
Zentralblatt MATH: 0851.62020
Digital Object Identifier: doi:10.1080/01621459.1995.10476592 - Laud, P. W. and Ibrahim, J. G. (1995). Predictive model selection. J. Roy. Statist. Soc. Ser. B 57 247–262.Mathematical Reviews (MathSciNet): MR1325389
- Liang, F., Paulo, R., Molina, G., Clyde, M. A. and Berger, J. O. (2008). Mixtures of $g$ priors for Bayesian variable selection. J. Amer. Statist. Assoc. 103 410–423.Mathematical Reviews (MathSciNet): MR2420243
Zentralblatt MATH: 05564499
Digital Object Identifier: doi:10.1198/016214507000001337 - Maruyama, Y. and George, E. I. (2008). gBF: A fully Bayes factor with a generalized $g$-prior. Available at arXiv:0801.4410v2 [stat.ME].Mathematical Reviews (MathSciNet): MR2906885
Zentralblatt MATH: 1231.62036
Digital Object Identifier: doi:10.1214/11-AOS917
Project Euclid: euclid.aos/1324563354 - Maruyama, Y. and Strawderman, W. E. (2010). Robust Bayesian variable selection with sub-harmonic priors. Available at arXiv:1009.1926v2 [stat.ME].
- Moreno, E., Bertolino, F. and Racugno, W. (1998). An intrinsic limiting procedure for model selection and hypotheses testing. J. Amer. Statist. Assoc. 93 1451–1460.Mathematical Reviews (MathSciNet): MR1666640
Zentralblatt MATH: 1064.62513
Digital Object Identifier: doi:10.1080/01621459.1998.10473805 - Pérez, J. M. and Berger, J. O. (2002). Expected-posterior prior distributions for model selection. Biometrika 89 491–511.Mathematical Reviews (MathSciNet): MR1929158
Zentralblatt MATH: 1036.62026
Digital Object Identifier: doi:10.1093/biomet/89.3.491 - Robert, C. P., Chopin, N. and Rousseau, J. (2009). Harold Jeffreys’s theory of probability revisited. Statist. Sci. 24 141–172.Mathematical Reviews (MathSciNet): MR2655841
Digital Object Identifier: doi:10.1214/09-STS284
Project Euclid: euclid.ss/1263478373 - Scott, J. G. and Berger, J. O. (2010). Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. Ann. Statist. 38 2587–2619.Mathematical Reviews (MathSciNet): MR2722450
Zentralblatt MATH: 1200.62020
Digital Object Identifier: doi:10.1214/10-AOS792
Project Euclid: euclid.aos/1278861454 - Spiegelhalter, D. J. and Smith, A. F. M. (1982). Bayes factors for linear and log-linear models with vague prior information. J. Roy. Statist. Soc. Ser. B 44 377–387.Mathematical Reviews (MathSciNet): MR693237
- Strawderman, W. E. (1971). Proper Bayes minimax estimators of the multivariate normal mean. Ann. Math. Statist. 42 385–388.Mathematical Reviews (MathSciNet): MR397939
Zentralblatt MATH: 0222.62006
Digital Object Identifier: doi:10.1214/aoms/1177693528
Project Euclid: euclid.aoms/1177693528 - Suzuki, Y. (1983). On Bayesian approach to model selection. In Proceedings of the International Statistical Institute 288–291. ISI Publications, Voorburg.
- Weisstein, E. W. (2009). Appell hypergeometric function from mathworld—a Wolfram web resource. Available at http://mathworld.wolfram.com/AppellHypergeometricFunction.html.
- Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with $g$-prior distributions. In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti (A. Zellner, ed.) 389–399. North-Holland, Amsterdam.
- Zellner, A. and Siow, A. (1980). Posterior odds ratio for selected regression hypotheses. In Bayesian Statistics 1 (J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 585–603. Univeristy Press, Valencia.Mathematical Reviews (MathSciNet): MR638871
- Zellner, A. and Siow, A. (1984). Basic Issues in Econometrics. University of Chicago Press, Chicago.
- You have access to this content.
- You have partial access to this content.
- You do not have access to this content.
More like this
- Context-Dependent Score Based Bayesian Information Criteria
Underhill, N. T. and Smith, J. Q., Bayesian Analysis, 2016 - Bayesian Experimental Design: A Review
Chaloner, Kathryn and Verdinelli, Isabella, Statistical Science, 1995 - Objective Priors: An Introduction for Frequentists
Ghosh, Malay, Statistical Science, 2011
- Context-Dependent Score Based Bayesian Information Criteria
Underhill, N. T. and Smith, J. Q., Bayesian Analysis, 2016 - Bayesian Experimental Design: A Review
Chaloner, Kathryn and Verdinelli, Isabella, Statistical Science, 1995 - Objective Priors: An Introduction for Frequentists
Ghosh, Malay, Statistical Science, 2011 - Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
Breheny, Patrick and Huang, Jian, Annals of Applied Statistics, 2011 - New formulation of the logistic-Gaussian process to analyze trajectory tracking data
Mastrantonio, Gianluca, Grazian, Clara, Mancinelli, Sara, and Bibbona, Enrico, Annals of Applied Statistics, 2019 - Variable Selection via Penalized Credible Regions with Dirichlet–Laplace Global-Local Shrinkage Priors
Zhang, Yan and Bondell, Howard D., Bayesian Analysis, 2018 - Variable selection for partially linear models via learning gradients
Yang, Lei, Fang, Yixin, Wang, Junhui, and Shao, Yongzhao, Electronic Journal of Statistics, 2017 - Model selection in overlapping stochastic block models
Latouche, Pierre, Birmelé, Etienne, and Ambroise, Christophe, Electronic Journal of Statistics, 2014 - Adaptive Variable Selection for Sequential Prediction in Multivariate Dynamic Models
Lavine, Isaac, Lindon, Michael, and West, Mike, Bayesian Analysis, 2020 - Clustering and variable selection for categorical multivariate data
Bontemps, Dominique and Toussile, Wilson, Electronic Journal of Statistics, 2013