Bayesian Analysis

Prior Distributions for Objective Bayesian Analysis

Guido Consonni, Dimitris Fouskakis, Brunero Liseo, and Ioannis Ntzoufras

Full-text: Open access

Abstract

We provide a review of prior distributions for objective Bayesian analysis. We start by examining some foundational issues and then organize our exposition into priors for: i) estimation or prediction; ii) model selection; iii) high-dimensional models. With regard to i), we present some basic notions, and then move to more recent contributions on discrete parameter space, hierarchical models, nonparametric models, and penalizing complexity priors. Point ii) is the focus of this paper: it discusses principles for objective Bayesian model comparison, and singles out some major concepts for building priors, which are subsequently illustrated in some detail for the classic problem of variable selection in normal linear models. We also present some recent contributions in the area of objective priors on model space. With regard to point iii) we only provide a short summary of some default priors for high-dimensional models, a rapidly growing area of research.

Article information

Source
Bayesian Anal., Volume 13, Number 2 (2018), 627-679.

Dates
First available in Project Euclid: 14 April 2018

Permanent link to this document
https://projecteuclid.org/euclid.ba/1523671250

Digital Object Identifier
doi:10.1214/18-BA1103

Mathematical Reviews number (MathSciNet)
MR3807861

Zentralblatt MATH identifier
06989961

Subjects
Primary: 62F15: Bayesian inference 62-02: Research exposition (monographs, survey articles)
Secondary: 62J05: Linear regression 62A01: Foundations and philosophical topics

Keywords
objective Bayes model comparison criteria for model choice noninformative prior reference prior variable selection high-dimensional model

Rights
Creative Commons Attribution 4.0 International License.

Citation

Consonni, Guido; Fouskakis, Dimitris; Liseo, Brunero; Ntzoufras, Ioannis. Prior Distributions for Objective Bayesian Analysis. Bayesian Anal. 13 (2018), no. 2, 627--679. doi:10.1214/18-BA1103. https://projecteuclid.org/euclid.ba/1523671250


Export citation

References

  • Altomare, D., Consonni, G., and La Rocca, L. (2013). “Objective Bayesian Search of Gaussian Directed Acyclic Graphical Models for Ordered Variables with Non-Local Priors.” Biometrics, 69: 478–487.
  • Arima, S., Datta, G. S., and Liseo, B. (2012). “Objective Bayesian analysis of a measurement error small area model.” Bayesian Analysis, 7: 363–383.
  • Barbieri, M. and Berger, J. (2004). “Optimal predictive model selection.” The Annals of Statistics, 32: 870–897.
  • Bayarri, M. J., Berger, J. O., Forte, A., and García-Donato, G. (2012). “Criteria for Bayesian model choice with application to variable selection.” The Annals of Statistics, 40: 1550–1577.
  • Bayarri, M. J., Berger, J. O., and Pericchi, L. R. (2014). “The Effective Sample Size.” Econometric Reviews, 33(1–4): 197–217.
  • Bayarri, M. J. and García-Donato, G. (2007). “Extending conventional priors for testing general hypotheses in linear models.” Biometrika, 94: 135–152.
  • Bayarri, M. J. and García-Donato, G. (2008). “Generalization of Jeffreys divergence-based priors for Bayesian hypothesis testing.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70: 981–1003.
  • Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., Cesarini, D., Chambers, C. D., Clyde, M., Cook, T. D., Boeck, P. D., Dienes, Z., Dreber, A., Easwaran, K., Efferson, C., Fehr, E., Fidler, F., Field, A. P., Forster, M., George, E. I., Gonzalez, R., Goodman, S., Green, E., Green, D. P., Greenwald, A., Hadfield, J. D., Hedges, L. V., Held, L., Ho, T. H., Hoijtink, H., Jones, J. H., Hruschka, D. J., Imai, K., Imbens, G., Ioannidis, J. P., Jeon, M., Kirchler, M., Laibson, D., List, J., Little, R., Lupia, A., Machery, E., Maxwell, S. E., McCarthy, M., Moore, D., Morgan, S. L., Munafó, M., Nakagawa, S., Nyhan, B., Parker, T. H., Pericchi, L. R., Perugini, M., Rouder, J., Rousseau, J., Savalei, V., Schönbrodt, F. D., Sellke, T., Sinclair, B., Tingley, D., Zandt, T. V., Vazire, S., Watts, D. J., Winship, C., Wolpert, R. L., Xie, Y., Young, C., Zinman, J., and Johnson, V. E. (2017). “Redefine Statistical Significance.” Nature Human Behavior.
  • Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed.. Springer, New York.
  • Berger, J. O. (2006). “The case for objective Bayesian analysis.” Bayesian Analysis, 1: 385–402.
  • Berger, J. O. and Bernardo, J. M. (1989). “Estimating a product of means: Bayesian analysis with reference priors.” Journal of the American Statistical Association, 84: 200–207.
  • Berger, J. O. and Bernardo, J. M. (1992). “Ordered group reference priors with application to the multinomial problem.” Biometrika, 79: 25.
  • Berger, J. O., Bernardo, J. M., and Mendoza, M. (1989). “On priors that maximize expected information.” In Klein, J. P. and Lee, J. C. (eds.), Recent Developments in Statistics and their Applications, 1–20. Seoul: Freedom Academy Publishing.
  • Berger, J. O., Bernardo, J. M., and Sun, D. (2009). “The formal definition of reference priors.” Annals of Statistics, 37: 905–938.
  • Berger, J. O., Bernardo, J. M., and Sun, D. (2012). “Objective priors for discrete parameter spaces.” Journal of the American Statistical Association, 107: 636–648.
  • Berger, J. O., Bernardo, J. M., and Sun, D. (2015). “Overall Objective Priors (with discussion).” Bayesian Analysis, 10: 189–221.
  • Berger, J. O., De Oliveira, V., and Sansò, B. (2001). “Objective Bayesian Analysis of Spatially Correlated Data.” Journal of the American Statistical Association, 96: 1361–1374.
  • Berger, J. O. and Mortera, J. (1999). “Default Bayes Factors for Nonnested Hypothesis Testing.” Journal of the American Statistical Association, 94: 542–554.
  • Berger, J. O. and Pericchi, L. R. (1996). “The Intrinsic Bayes Factor for Model Selection and Prediction.” Journal of the American Statistical Association, 91: 109–122.
  • Berger, J. O. and Pericchi, L. R. (2001). “Objective Bayesian methods for model selection: Introduction and comparison.” In Model Selection. Institute of Mathematical Statistics Lecture Notes, Monograph Series 38, 135–207. IMS, Beachwood, OH.
  • Berger, J. O. and Pericchi, L. R. (2004). “Training Samples in Objective Model Selection.” The Annals of Statistics, 32: 841–869.
  • Berger, J. O., Pericchi, L. R., and Varshavsky, J. (1998). “Bayes factors and marginal distributions in invariant situations.” Sankhya, 60: 109–122.
  • Berger, J. O., Strawderman, W., and Tang, D. (2005). “Posterior Propriety and Admissibility of Hyperpriors in Normal Hierarchical Models.” The Annals of Statistics, 33: 606–646.
  • Berger, J. O. and Strawderman, W. E. (1996). “Choice of hierarchical priors: Admissibility in estimation of normal means.” The Annals of Statistics, 24: 931–951.
  • Berger, J. O. and Sun, D. (2008). “Objective priors for the bivariate normal model.” The Annals of Statistics, 36: 963–982.
  • Bernardo, J. M. (1979). “Reference posterior distributions for Bayesian inference (with discussion).” Journal of the Royal Statistical Society B, 2: 113–147.
  • Bernardo, J. M. and Rueda, R. (2002). “Bayesian Hypothesis Testing: A Reference Approach.” International Statistical Review, 70: 351–372.
  • Bernardo, J. M. and Smith, A. (1994). Bayesian Theory. Chichester, UK: Wiley.
  • Bertolino, F. and Racugno, W. (2000). “Bayesian model selection approach to analysis of variance under heteroscedasticity.” 49: 503–517.
  • Bhadra, A., Datta, J., Polson, N. G., and Willard, B. (2016). “Default Bayesian analysis with global-local shrinkage priors.” Biometrika, 103(4): 955–969.
  • Bhattacharya, A., Pati, D., Pillai, N., and Dunson, D. (2015). “Dirichlet-Laplace priors for Optimal Shrinkage.” Journal of the American Statistical Association, 110: 1479–1490.
  • Bhattacharya, A., Pati, D., and Yang, Y. (2016). “Bayesian fractional posteriors.” arXiv:1611.01125.
  • Bodnar, O., Link, A., and Elster, C. (2016). “Objective Bayesian inference for a generalized marginal random effects model.” Bayesian Analysis, 11: 25–45.
  • Branco, M. D., Genton, M. G., and Liseo, B. (2013). “Objective Bayesian analysis of skew-$t$ distributions.” Scandinavian Journal of Statistics. Theory and Applications, 40: 63–85.
  • Brown, L. D. (1971). “Admissible Estimators, Recurrent Diffusions, and Insoluble Boundary Value Problems.” The Annals of Mathematical Statistics, 42: 855–903.
  • Bush, C. A., Lee, J., and MacEachern, S. N. (2010). “Minimally informative prior distributions for non-parametric Bayesian analysis.” Journal of the Royal Statistical Society B, 72: 253–268.
  • Cano, J. and Salmerón, D. (2013). “Integral Priors and Constrained Imaginary Training Samples for Nested and Non-Nested Bayesian Model Comparison.” Bayesian Analysis, 8: 361–380.
  • Cano, J. A., Kessler, M., and Moreno, E. (2004). “On Intrinsic Priors for Nonnested Models.” TEST, 13: 445–463.
  • Carvalho, C., Polson, N., and Scott, J. (2010). “The horseshoe estimator for sparse signal.” Biometrika, 97: 465–480.
  • Carvalho, C. and Scott, J. (2009). “Objective Bayesian model selection in Gaussian graphical models.” Biometrika, 96: 497–512.
  • Casella, G., Girón, F. J., Martínez, M. L., and Moreno, E. (2009). “Consistency of Bayesian Procedures for Variable Selection.” The Annals of Statistics, 37: 1207–1228.
  • Casella, G. and Moreno, E. (2006). “Objective Bayesian Variable Selection.” Journal of the American Statistical Association, 101: 157–167.
  • Casella, G. and Moreno, E. (2009). “Assessing Robustness of Intrinsic Tests of Independence in Two-Way Contingency Tables.” Journal of the American Statistical Association, 104: 1261–1271.
  • Castillo, I. and Misner, R. (2018). “Empirical Bayes analysis of spike and slab posterior distributions.” arXiv:1801.01696v1.
  • Castillo, I., Schmidt-Hieber, J., and van der Vaart, A. (2015). “Bayesian linear regression with sparse priors.” The Annals of Statistics, 43(5): 1986–2018.
  • Castillo, I. and van der Vaart, A. (2012). “Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences.” The Annals of Statistics, 40: 2069–2101.
  • Chen, M., Ibrahim, J. G., and Shao, Q. M. (2000). “Power prior distributions for generalized linear models.” Journal of Statistical Planning and Inference, 84: 121–137.
  • Chib, S. and Kuffner, T. A. (2016). “Bayes factor consistency.” arXiv:1607.00292.
  • Chipman, H., George, E., and McCulloch, R. (2001). “The practical implementation of Bayesian model selection.” Model Selection, IMS Lecture Notes – Monograph Series, 38: 67–116.
  • Choirat, C. and Seri, R. (2012). “Estimation in discrete parameter models.” Statistical Science, 27: 278–293.
  • Clyde, M. and Iversen, E. (2013). “Bayesian Model Averaging in the M-open framework.” In Bayesian Theory and Applications, in P. Damien, P. Dellaportas, N.G. Polson, and D.A. Stephens, eds., 484–498. Oxford University Press.
  • Consonni, G., Forster, J. J., and La Rocca, L. (2013). “The Whetstone and the Alum Block: Balanced Objective Bayesian Comparison of Nested Models for Discrete Data.” Statistical Science, 28: 398–423.
  • Consonni, G. and La Rocca, L. (2008). “Tests Based on Intrinsic Priors for the Equality of Two Correlated Proportions.” Journal of the American Statistical Association, 103: 1260–1269.
  • Consonni, G. and La Rocca, L. (2011). “On moment priors for Bayesian model choice with applications to directed acyclic graphs.” In Bernardo, J. M., Bayarri, M. J., Berger, J. O., Dawid, A., Heckerman, D., Smith, A. F. M., and West, M. (eds.), Bayesian Statistics, 119–144. Oxford University Press.
  • Consonni, G. and La Rocca, L. (2012). “Objective Bayes Factors for Gaussian Directed Acyclic Graphical Models.” Scandinavian Journal of Statistics, 39: 743–756.
  • Consonni, G., La Rocca, L., and Peluso, S. (2017). “Objective Bayes Covariate-Adjusted Sparse Graphical Model Selection.” Scandinavian Journal of Statistics, 44: 741–764.
  • Consonni, G., Moreno, E., and Venturini, S. (2011). “Testing Hardy-Weinberg equilibrium: An objective Bayesian analysis.” Statistics in Medicine, 30: 62–74.
  • Consonni, G. and Paroli, R. (2017). “Objective Bayesian Comparison of Constrained Analysis of Variance Models.” Psychometrika, 82: 589–609.
  • Consonni, G. and Veronese, P. (2008). “Compatibility of Prior Specifications Across Linear Models.” Statistical Science, 23: 332–353.
  • Corander, J. and Villani, M. (2004). “Bayesian assessment of dimensionality in reduced rank regression.” Statistica Neerlandica, 58: 255–270.
  • Corander, J. and Villani, M. (2006). “A Bayesian Approach to Modelling Graphical Vector Autoregressions.” Journal of Time Series Analysis, 27: 141–156.
  • Cui, W. and George, E. I. E. (2008). “Empirical Bayes vs. fully Bayes variable selection.” Journal of Statistical Planning and Inference, 138: 888–900.
  • Datta, G. and Rao, J. (2010). “The Choice of Nonsubjective Priors on Hyperparameters for Hierarchical Bayes Models.” In Chen, M., Mueller, P., Sun, D., Ye, K., and Dey, D. (eds.), Frontiers of Statistical Decision Making and Bayesian Analysis: In Honor of Jim Berger, 237–246. Springer, New York.
  • Datta, G. S. and Mukerjee, R. (2004). Probability Matching Priors: Higher Order Asymptotics. Lecture Notes in Statistics. Sringer, New York.
  • Datta, J. and Ghosh, J. K. (2013). “Asymptotic Properties of Bayes Risk for the Horseshoe Prior.” Bayesian Analysis, 8: 111–132.
  • Dawid, A. (1982). “Intersubjective Statistical Models.” In Exchangeability in Probability and Statistics, in G. Koch and F. Spizzichino eds., 217–232. Amsterdam, North Holland.
  • Dawid, A., Stone, M., and Zidek, J. (1973). “Marginalization Paradoxes in Bayesian and Structural Inference (with discussion).” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 35: 189–233.
  • Dawid, A. P. (2006). “Invariant Prior Distributions.” In Encyclopedia of Statistical Sciences. Wiley, New York.
  • Dawid, A. P. (2011). “Posterior Model Probabilities.” In Bandyopadhyay, P. S. and Forster, M. (eds.), Philosophy of Statistics, 607–630. Elsevier.
  • Dawid, A. P. and Lauritzen, S. (2011). “Compatible prior distributions.” In George, E. I. (ed.), Bayesian Methods with Applications to Science, Policy and Official Statistics. Proceedings of the 6th World Meeting, 109–118. International Society for Bayesian Analysis, Office for Official Publications of the European Communities.
  • Dawid, A. P. and Musio, M. (2015). “Bayesian Model Selection Based on Proper Scoring Rules.” Bayesian Analysis, 10: 479–499.
  • De Blasi, P., Favaro, S., Lijoi, A., Mena, R., Prunster, I., and Ruggiero, M. (2015). “Are Gibbs-Type Priors the Most Natural Generalization of the Dirichlet Process?” IEEE Tranactions on Pattern Analysis and Machine Intelligence, 37,2: 803–821.
  • De-Santis, F. and Spezzaferri, F. (1997). “Alternative Bayes factors for model selection.” Canadian Journal of Statistics, 25: 503–515.
  • Dellaportas, P., Forster, J. J., and Ntzoufras, I. (2012). “Joint Specification of Model Space and Parameter Space Prior Distributions.” Statistical Science, 27: 232–246.
  • Dette, H., Ley, C., and Rubio, F. (2017). “Natural (Non-)Informative Priors for Skew-symmetric Distributions (to appear).” Scandinavian Journal of Statistics.
  • Fernández, C., Ley, E., and Steel, M. F. J. (2001). “Benchmark Priors For Bayesian Model Averaging.” Journal of Econometrics, 100: 381–427.
  • Fonseca, T., Ferreira, M., and Migon, H. (2008). “Objective Bayesian analysis for the Student-t regression model.” Biometrika, 95: 325–333.
  • Foster, D. P. and George, E. I. (1994). “The Risk Inflation Criterion for Multiple Regression.” Annals of Statistics, 22: 1947–1975.
  • Fouskakis, D. and Ntzoufras, I. (2016a). “Limiting behavior of the Jeffreys Power-Expected-Posterior Bayes Factor in Gaussian Linear Models.” Brazilian Journal of Probability and Statistics, 30: 299–320.
  • Fouskakis, D. and Ntzoufras, I. (2016b). “Power-Conditional-Expected Priors: Using $g$-priors with Random Imaginary Data for Variable Selection.” Journal of Computational and Graphical Statistics, 25: 647–664.
  • Fouskakis, D. and Ntzoufras, I. (2017). “Information Consistency of the Jeffreys Power-Expected-Posterior Prior in Gaussian Linear Models.” Metron, 75: 371–380.
  • Fouskakis, D., Ntzoufras, I., and Draper, D. (2009). “Bayesian variable selection using cost-adjusted BIC, with application to cost-effective measurement of quality of health care.” The Annals of Applied Statistics, 3: 663–690.
  • Fouskakis, D., Ntzoufras, I., and Draper, D. (2015). “Power-Expected-Posterior Priors for variable selection in Gaussian Linear Models.” Bayesian Analysis, 10: 75–107.
  • Fouskakis, D., Ntzoufras, I., and Pericchi, L. R. (2017a). “Priors via Imaginary Training Samples of Sufficient Statistics for Objective Bayesian Model Comparison.” (submitted); Technical Report, Dept. of Mathematics, National Technical University of Athens.
  • Fouskakis, D., Ntzoufras, I., and Perrakis, K. (2017b). “Power-Expected-Posterior Priors for Generalized Linear Models.” Bayesian Analysis (to appear).
  • Garcia-Donato, G. and Sun, D. (2007). “Objective Priors for Hypothesis Testing in One-Way Random Effects Models.” The Canadian Journal of Statistics, 35: 303–320.
  • Gelman, A. (2006). “‘Prior distributions for variance parameters in hierarchical models’.” Bayesian Analysis, 1: 515–533.
  • Gelman, A. and Hennig, C. (2017). “Beyond subjective and objective in statistics.” Journal of the Royal Statistical Society: Series A (Statistics in Society), 180(4): 967–1033.
  • George, E. I. (1999). “Discussion of “Bayesian Model Averaging and Model Search Strategies” by Clyde M.” In Bayesian Statistics, Vol. 6, in J. Bernardo, J. Berger, A. Dawid, and A. Smith, eds., 175–177. Oxford University Press.
  • George, E. I. (2010). “Dilution priors: Compensating for model space redundancy.” Borrowing Strength: Theory Powering Applications – A Festschrift for Lawrence D. Brown, IMS Collections, 6: 158–165.
  • George, E. I. and Foster, D. (2000). “Calibration and empirical Bayes variable selection.” Biometrika, 87: 731–747.
  • George, E. I. and Maruyama, Y. (2014). “Posterior Odds with a Generalized Hyper-g-Prior.” Econometric Reviews, 33: 251–269.
  • George, E. I. and McCulloch, R. E. (1993). “Variable Selection via Gibbs Sampling.” Journal of the American Statistical Association, 88: 881–889.
  • Ghosh, M. (2011). “‘Objective Priors: An Introduction for Frequentists.” Statistical Science, 26: 187–202.
  • Girón, F. J., Martínez, M. L., Moreno, E., and Torres, F. (2006). “Objective testing procedures in Linear Models: Calibration of the p-values.” Scandinavian Journal of Statistics, 33: 765–784.
  • Girón, F. J., Moreno, E., and Casella, G. (2007). “Objective Bayesian analysis of multiple changepoints for linear models.” In Bayesian Statistics 8, J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West (eds)., 227–252. Oxford University Press, Oxford.
  • Good, I. (1950). Probability and the Weighting of Evidence. London, UK: Charles Griffin.
  • Good, I. (1958). “Significance Tests in Parallel and in Series.” Journal of the American Statistical Association, 53: 799–813.
  • Goutis, C. and Robert, C. (1998). “Model Choice in Generalised Linear Models: A Bayesian Approach Via Kullback–Leibler Projections.” Biometrika, 85(1): 29–37.
  • Griffin, J. E. and Brown, P. J. (2010). “Inference with normal-gamma prior distributions in regression problems.” Bayesian Analysis, 5: 171–188.
  • Guillotte, S. and Perron, F. (2012). “Bayesian estimation of a bivariate copula using the Jeffreys prior.” Bernoulli, 18: 496–519.
  • Han, C. and Chaloner, K. (2004). “Bayesian Experimental Design for Nonlinear Mixed-Effects Models with Application to HIV Dynamics.” Biometrics, 60(1): 25–33.
  • Hansen, M. H. and Yu, B. (2001). “Model selection and the principle of minimum description length.” Journal of the American Statistical Association, 96: 746–774.
  • Held, L., Sabanés Bové, D., and Gravestock, I. (2015). “Approximate Bayesian Model Selection with the Deviance Statistic.” Statistical Science, 30: 242–257.
  • Hoeting, J., Madigan, D., and Raftery, A. (1996). “A Method for Simultaneous Variable Selection and Outlier Identification in Linear Regression.” Computational Statistics and Data Analysis, 22: 251–270.
  • Hoffmann, M., Rousseau, J., and Schmidt-Hieber, J. (2015). “On adaptive posterior concentration rates.” The Annals of Statistics, 43: 2259–2295.
  • Hu, J. and Johnson, V. E. (2009). “Bayesian model selection using test statistics.” Journal of the Royal Statistical Society B, 71: 143–158.
  • Ibrahim, J. G. and Chen, M. H. (2000). “Power Prior Distributions for Regression Models.” Statistical Science, 15: 46–60.
  • Ishwaran, H. and Rao, J. (2005). “Spike and Slab Variable Selection: Frequentist and Bayesian Strategies.” The Annals of Statistics, 33: 730–773.
  • Iwaki, K. (1997). “Posterior Expected Marginal Likelihood for Testing Hypotheses.” Journal of Economics, Asia University, 21: 105–134.
  • Jaynes, E. (2003). Probability Theory. Cambridge University Press.
  • Jeffreys, H. (1961). Theory of Probability (3rd edition). Oxford University Press.
  • Johnson, V. and Rossell, D. (2010). “On the use of non-local prior densities in Bayesian hypothesis tests.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72: 143–170.
  • Johnson, V. E. (2005). “Bayes factors based on test statistics.” Journal of the Royal Statistical Society B, 67: 689–701.
  • Johnson, V. E. (2013). “Uniformly most powerful Bayesian tests.” The Annals of Statistics, 41: 1716–1741.
  • Kamary, K., Mengersen, K., Robert, C. P., and Rousseau, J. (2014). “Testing hypotheses via a mixture estimation model.” arXiv:1412.2044.
  • Kass, R. and Raftery, A. (1995). “Bayes Factors.” Journal of the American Statistical Association, 90: 773–795.
  • Kass, R. E. and Wasserman, L. (1995). “A Reference Bayesian Test for Nested Hypotheses and its Relationship to the Schwarz Criterion.” Journal of the American Statistical Association, 90: 928–934.
  • Kass, R. E. and Wasserman, L. (1996). “The Selection of Prior Distributions by Formal Rules.” Journal of the American Statistical Association, 91: 1343–1369.
  • Kim, S. W. and Sun, D. (2000). “Intrinsic priors for model selection using an encompassing model with applications to censored failure time data.” Lifetime Data Analysis, 6: 251–269.
  • Lafferty, J. D. and Wasserman, L. A. (2013). “Iterative Markov Chain Monte Carlo Computation of Reference Priors and Minimax Risk.” CoRR, abs/1301.2286.
  • Lee, J., MacEachern, S. N., Lu, Y., and Mills, G. B. (2014). “Local-mass preserving prior distributions for nonparametric Bayesian models.” Bayesian Analysis, 9: 307–330.
  • Leisen, F., Villa, C., and Walker, S. (2017). “On a Global Objective Prior from Score Rule.” (submitted); Arxiv: https://arxiv.org/pdf/1706.00599.pdf.
  • Leon-Novelo, L., Moreno, E., and Casella, G. (2012). “Objective Bayes model selection in probit models.” Statistics in Medicine, 31: 353–365.
  • Leppä-aho, J., Pensar, J., Roos, T., and Corander, J. (2016). “Learning Gaussian Graphical Models With Fractional Marginal Pseudo-likelihood.” arXiv:1602.07863.
  • Ley, E. and Steel, M. (2012). “Mixtures of g-priors for Bayesian Model Averaging with Economic Applications.” Journal of Econometrics, 171: 251–266.
  • Li, Y. (2013). “Bayesian Hierarchical Models for Model Choice.” Ph.D. thesis, Department of Statistical Science, Duke University, USA.
  • Li, Y. and Clyde, M. A. (2016). “Mixtures of $g$-priors in generalized linear models.” arXiv:1503.06913v2.
  • Liang, F., Paulo, R., Molina, G., Clyde, M. A., and Berger, J. O. (2008). “Mixtures of g-Priors for Bayesian Variable Selection.” Journal of the American Statistical Association, 103: 410–423.
  • Lindley, D. (1957). “A Statistical Paradox.” Biometrika, 44: 187–192.
  • Liseo, B. and Macaro, C. (2013). “Objective priors for causal AR($p$) with partial autocorrelations.” Journal of Statistical Computation and Simulation, 83: 1613–1628.
  • Malesios, C., Demiris, N., Kalogeropoulos, K., and Ntzoufras, I. (2017). “Bayesian spatio-temporal epidemic models with applications to sheep pox.” Statistics in Medicine (to appear).
  • Maruyama, Y. and George, E. I. (2011). “Fully Bayes factors with a generalized $g$-prior.” The Annals of Statistics, 39: 2740–2765.
  • M’lan, C. E. and Chen, M.-H. (2015). “Objective Bayesian inference for bilateral data.” Bayesian Analysis, 10: 139–170.
  • Moreno, E. (1997). “Bayes factors for intrinsic and fractional priors in nested models. Bayesian robustness.” Lecture Notes-Monograph Series, 31.
  • Moreno, E. (2005). “Objective Bayesian methods for one-sided testing.” TEST, 14: 181–198.
  • Moreno, E., Bertolino, F., and Racugno, W. (1998). “An Intrinsic Limiting Procedure for Model Selection and Hypotheses Testing.” Journal of the American Statistical Association, 93: 1451–1460.
  • Moreno, E. and Girón, F. J. (2008). “Comparison of Bayesian Objective Procedures for Variable Selection in Linear Regression.” TEST, 17: 472–490.
  • Moreno, E., Girón, F. J., and Casella, G. (2010). “Consistency of objective Bayes factors as the model dimension grows.” Annals of Statistics, 38: 1937–1952.
  • Moreno, E. and Liseo, B. (2003). “A default Bayesian test for the number of components in a mixture.” Journal of Statistical Planning and Inference, 111: 129–142.
  • Moreno, E. and Pericchi, L. R. (2014). “Intrinsic Priors for Objective Bayesian Model Selection.” In Bayesian Model Comparison, 279–300. Emerald Group Publishing Limited.
  • Moreno, E., Torres, F., and Casella, G. (2005). “Testing equality of regression coefficients in heteroscedastic normal regression models.” Journal of Statistical Planning and Inference, 131: 117–134.
  • Mukhopadhyay, M. and Minerva, T. (2017). “A mixture of g-priors for variable selection when the number of regressors grows with the sample size.” TEST, 26: 377–404.
  • Mulder, J. and Wagenmakers, E.-J. (2016). “Editors’ introduction to the special issue “Bayes factors for testing hypotheses in psychological research: Practical relevance and new developments”.” Journal of Mathematical Psychology, 72: 1–5.
  • Müller, P. and Mitra, R. (2013). “Bayesian nonparametric inference—why and how.” Bayesian Analysis, 8: 269–302.
  • Neal, R. M. (2001). “Transferring Prior Information Between Models Using Imaginary Data.” Technical Report 0108, Department of Statistics and Department of Computer Science University of Toronto, Canada.
  • Ntzoufras, I. (2009). Bayesian Modeling Using WinBUGS. Wiley Series in Computational Statistics. Hoboken, NJ: Wiley and Sons.
  • Ntzoufras, I., Dellaportas, P., and Forster, J. J. (2003). “Bayesian Variable and Link Determination for Generalized Linear Models.” Journal of Statistical Planning and Inference, 111: 165–180.
  • Ntzoufras, I. and Tarantola, C. (2008). “Bayesian Analysis of Marginal Log-Linear Graphical Models for Three Way Contingency Tables.” arXiv:0807.1001.
  • O’Hagan, A. (1995). “Fractional Bayes Factors for Model Comparison.” Journal of the Royal Statistical Society B, 57: 99–138.
  • Overstall, A. M. and Forster, J. J. (2010). “Default Bayesian model determination methods for generalised linear mixed models.” Computational Statistics & Data Analysis, 54: 3269–3288.
  • Pérez, J. (1998). “Development of Expected Posterior Prior Distribution for Model Comparisons.” Ph.D. thesis, Department of Statistics, Purdue University, USA.
  • Pérez, J. M. and Berger, J. O. (2002). “Expected-posterior Prior Distributions for Model Selection.” Biometrika, 89: 491–511.
  • Pérez, M., Pericchi, L. R., and Ramirez, I. (2017). “The Scaled Beta2 Distribution as a Robust Prior for Scales).” Bayesian Analysis, 12: 615–637.
  • Pericchi, L. R. (2005). “Model Selection and Hypothesis Testing based on Objective Probabilities and Bayes Factors.” In Dey, D. and Rao, C. (eds.), Bayesian Thinking Modeling and Computation, volume 25 of Handbook of Statistics, 115–149. Elsevier.
  • Piironen, J. and Vehtari, A. (2017a). “On the Hyperprior Choice for the Global Shrinkage Parameter in the Horseshoe Prior.” In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS, PMLR 54, 905–913.
  • Piironen, J. and Vehtari, A. (2017b). “Sparsity information and regularization in the horseshoe and other shrinkage priors.” Electronic Journal of Statistics, 11(2): 5018–5051.
  • Plummer, M. (2008). “Penalized loss functions for Bayesian model comparison.” Biostatistics, 9(3): 523–539.
  • Polson, N. and Scott, J. (2012a). “On the half-Cauchy prior for a global scale parameter.” Bayesian Analysis, 7: 887–902.
  • Polson, N. and Scott, J. G. (2012b). “Local shrinkage rules, Lévy processes and regularized regression.” Journal of the Royal Statistical Society B, 74: 287–311.
  • Rivoirard, V. and Rousseau, J. (2012). “Posterior concentration rates for infinite dimensional exponential families.” Bayesian Analysis, 7: 311–333.
  • Robert, C. (1993). “‘A Note on Jeffreys-Lindley Paradox’.” Statistica Sinica, 3: 601–608.
  • Robert, C. (2007). The Bayesian Choice. 2nd ed. New York: Springer-Verlag.
  • Robert, C. (2014). “Jeffreys prior with improper posterior[Blog post].” retrieved from https://xianblog.wordpress.com/2014/05/12/jeffreys-prior-with-improper-posterior/.
  • Ročková, V. (2018). “Bayesian estimation of sparse signals with a continuous spike-and-slab prior.” Annals of Statistics, 46(1): 401–437.
  • Ročková, V. and George, E. I. (2014). “EMVS: The EM Approach to Bayesian variable selection.” Journal of the American Statistical Association, 109: 828–846.
  • Ročková, V. and George, E. I. (2018). “The Spike-and-Slab LASSO.” Journal of the American Statistical Association (accepted).
  • Rossell, D. and Telesca, D. (2017). “Nonlocal Priors for High-Dimensional Estimation.” Journal of the American Statistical Association, 112(517): 254–265.
  • Rubin, H. (1987). “A weak system of axioms for “rational” behavior and the nonseparability of utility from prior.” Statistics & Decisions, 5: 47–58.
  • Rubio, F. J. and Liseo, B. (2014). “On the independence Jeffreys prior for skew-symmetric models.” Statistics & Probability Letters, 85: 91–97.
  • Sabanés Bové, D. and Held, L. (2011). “Hyper-$g$ Priors for Generalized Linear Models.” Bayesian Analysis, 6: 387–410.
  • Sabanés Bové, D., Held, L., and Kauermann, G. (2015). “Objective Bayesian Model Selection in Generalized Additive Models With Penalized Splines.” Journal of Computational and Graphical Statistics, 24: 394–415.
  • Sansó, B., Pericchi, L. R., and Moreno, E. (1996). “On the robustness of the intrinsic Bayes factor for nested models. (with discussion).” In Bayesian Robustness 2, In J. Berger, F. Ruggeri, and L. Wasserman (Eds.), 157–176. California, USA: IMS Monographs.
  • Savage, L. (1954). The Foundations of Statistical Inference. John Wiley.
  • Schwarz, G. (1978). “Estimating the Dimension of a Model.” The Annals of Statistics, 6: 461–464.
  • Scott, J. G. and Berger, J. O. (2010). “Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem.” The Annals of Statistics, 38: 2587–2619.
  • Simpson, D., Rue, H., Riebler, A., Martins, T., and Sorbye, S. (2017). “Penalising model component complexity: A principled, practical approach to constructing priors.” Statistical Science, 32(1): 1–28.
  • Som, A., Hans, C. M., and MacEachern, S. N. (2015). “Block Hyper-g Priors in Bayesian Regression.” arXiv:1406.6419v2.
  • Sorbye, S. and Rue, H. (2017). “Penalised complexity priors for stationary autoregressive processes.” Journal of Time Series Analysis, 38(6): 923–935.
  • Spiegelhalter, D. and Smith, A. (1980). “Bayes Factors for Linear and Log-linear Models with Vague Prior Information.” Journal of the Royal Statistical Society B, 44: 377–387.
  • Spiegelhalter, D. J. and Smith, A. F. M. (1982). “Bayes Factors for Linear and Log-Linear Models with Vague Prior Information.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 44(3): 377–387.
  • Spitzner, D. (2005). “Risk-reducing shrinkage estimation for generalized linear models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67: 183–196.
  • Spitzner, D. J. (2011). “Neutral-data comparisons for Bayesian testing.” Bayesian Analysis, 6(4): 603–638.
  • Stein, C. (1956). “Inadmissibility of the usual estimator for the mean of a multivariate distribution.” Proceedings of the Third Berkeley Symposium Mathematical Statistics and Probability, 1: 197–206.
  • Storey, J. (2003). “The positive false discovery rate: A Bayesian interpretation and the q-value.” The Annals of Statistics, 31: 2013–2035.
  • Sun, D., Tsutakawa, R. K., and He, Z. (2001). “Propriety of posteriors with improper priors in hierarchical linear mixed models.” Statistica Sinica, 11: 77–95.
  • Tansey, W., Koyejo, O., Poldrack, R., and Scott, J. (2018). “False discovery rate smoothing.” Journal of the American Statistical Association (accepted).
  • Torres-Ruiz, F., Moreno, E., and Girón, F. J. (2011). “Intrinsic priors for model comparison in multivariate normal regression.” Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales – Serie A: Matematicas, 105: 273–289.
  • Vallejos, C. A. and Steel, M. F. J. (2015). “Objective Bayesian survival analysis using shape mixtures of log-normal distributions.” Journal of the American Statistical Association, 110: 697–710.
  • van der Pas, S., Szabo, B., and var der Vaart, A. (2017). “Uncertainty Quantification for the Horseshoe (with Discussion).” Bayesian Analysis, 12: 1221–1274.
  • Villa, C. and Lee, J. E. (2015). “Model Prior Distribution for Variable Selection in Linear Regression Models.” arXiv:1512.08077.
  • Villa, C. and Walker, S. (2014a). “A cautionary note on the discrete uniform prior for the binomial N: Comment.” Ecology, 95: 2674–2677.
  • Villa, C. and Walker, S. (2014b). “Objective Prior for the Number of Degrees of Freedom of a Student $t$ Distribution.” Bayesian Analysis, 9: 197–220.
  • Villa, C. and Walker, S. (2015a). “An objective approach to prior mass functions for discrete parameter spaces.” Journal of the American Statistical Association, 110: 1072–1082.
  • Villa, C. and Walker, S. (2015b). “An Objective Bayesian Criterion to Determine Model Prior Probabilities.” Scandinavian Journal of Statistics, 42: 947–966.
  • Villa, C. and Walker, S. (2017). “On The Mathematics of The Jeffreys-Lindley Paradox.” Communications in Statistics – Theory and Methods, 46: 12290–12298.
  • Villani, M. (2001). “Fractional Bayesian Lag Length Inference in Multivariate Autoregressive Processes.” Journal of Time Series Analysis, 22: 67–86.
  • Walker, S., Damien, P., and Lenk, P. (2004). “On priors with a kullback-leibler property.” Journal of the American Statistical Association, 99(466): 404–408.
  • Wang, M. (2017). “Mixtures of $g$-priors for analysis of variance models with a diverging number of parameters.” Bayesian Analysis, 12: 511–532.
  • Wasserstein, R. L. and Lazar, N. A. (2016). “The ASA’s Statement on p-Values: CContext, Process, and Purpose.” The American Statistician, 70: 129–133.
  • Wetzels, R., Grasman, R. P., and Wagenmakers, E.-J. (2012). “A Default Bayesian Hypothesis Test for ANOVA Designs.” The American Statistician, 66: 104–111.
  • Wilson, M. A., Iversen, E. S., Clyde, M. A., Schmidler, S. C., and Schildkraut, J. M. (2010). “Bayesian model search and multilevel inference for SNP association studies.” Annals of Applied Statistics, 4: 1342–1364.
  • Womack, A. J., Fuentes, C., and Taylor-Rodriguez, D. (2015). “Model Space Priors for Objective Sparse Bayesian Regression.” arXiv:1511.04745.
  • Womack, A. J., Leon-Novelo, L., and Casella, G. (2014). “Inference from Intrinsic Bayes’ Procedures Under Model Selection and Uncertainty.” Journal of the American Statistical Association, 109: 1040–1053.
  • Xu, C., Sun, D., and He, C. (2014). “Objective Bayesian analysis for a capture-recapture model.” Annals of the Institute of Statistical Mathematics, 66: 245–278.
  • Ye, K. and Berger, J. O. (1991). “Noninformative Priors for Inferences in Exponential Regression Models.” Biometrika, 78: 645–656.
  • Yuan, M. and Lin, Y. (2005). “Efficient empirical Bayes variable selection and estimation in linear models.” Journal of the American Statistical Association, 100: 1215–1225.
  • Zellner, A. (1986). “On Assessing Prior Distributions and Bayesian Regression Analysis Using G-Prior distributions.” In Goel, P. and Zellner, A. (eds.), Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti, 233–243. Amsterdam: North-Holland.
  • Zellner, A. (2008). “Comments on Mixtures of g-priors for Bayesian Variable Selection.” Unpublished Report, Graduate School of Business, University of Chicago.
  • Zellner, A. and Siow, A. (1980). “Posterior Odds Ratios for Selected Regression Hypothesis (with discussion).” In Bernardo, J. M., DeGroot, M., Lindley, D., and Smith, A. (eds.), Bayesian Statistics 1, 585–606 & 618–647 (discussion). Oxford University Press.