Statistical Science

Approximate Bayesian Model Selection with the Deviance Statistic

Leonhard Held, Daniel Sabanés Bové, and Isaac Gravestock

Full-text: Open access

Enhanced PDF (718 KB)

Abstract

Bayesian model selection poses two main challenges: the specification of parameter priors for all models, and the computation of the resulting Bayes factors between models. There is now a large literature on automatic and objective parameter priors in the linear model. One important class are $g$-priors, which were recently extended from linear to generalized linear models (GLMs). We show that the resulting Bayes factors can be approximated by test-based Bayes factors (Johnson [ Scand. J. Stat. 35 (2008) 354–368]) using the deviance statistics of the models. To estimate the hyperparameter $g$, we propose empirical and fully Bayes approaches and link the former to minimum Bayes factors and shrinkage estimates from the literature. Furthermore, we describe how to approximate the corresponding posterior distribution of the regression coefficients based on the standard GLM output. We illustrate the approach with the development of a clinical prediction model for 30-day survival in the GUSTO-I trial using logistic regression.

Article information

Source
Statist. Sci., Volume 30, Number 2 (2015), 242-257.

Dates
First available in Project Euclid: 3 June 2015

Permanent link to this document
https://projecteuclid.org/euclid.ss/1433341481

Digital Object Identifier
doi:10.1214/14-STS510

Mathematical Reviews number (MathSciNet)
MR3353106

Zentralblatt MATH identifier
1332.62094

Keywords
Bayes factor deviance generalized linear model $g$-prior model selection shrinkage

Citation

Held, Leonhard; Sabanés Bové, Daniel; Gravestock, Isaac. Approximate Bayesian Model Selection with the Deviance Statistic. Statist. Sci. 30 (2015), no. 2, 242--257. doi:10.1214/14-STS510. https://projecteuclid.org/euclid.ss/1433341481


Export citation

References

  • Barbieri, M. M. and Berger, J. O. (2004). Optimal predictive model selection. Ann. Statist. 32 870–897.
  • Bayarri, M. J., Berger, J. O., Forte, A. and García-Donato, G. (2012). Criteria for Bayesian model choice with application to variable selection. Ann. Statist. 40 1550–1577.
  • Berger, J. O. and Pericchi, L. R. (2001). Objective Bayesian methods for model selection: Introduction and comparison. In Model Selection (P. Lahiri, eds.). Institute of Mathematical Statistics Lecture Notes—Monograph Series 38 135–207. IMS, Beachwood, OH.
  • Berger, J. O. and Sellke, T. (1987). Testing a point null hypothesis: Irreconcilability of $p$-values and evidence. J. Amer. Statist. Assoc. 82 112–139.
  • Claeskens, G. and Hjort, N. L. (2008). Model Selection and Model Averaging. Cambridge Univ. Press, Cambridge.
  • Copas, J. B. (1983). Regression, prediction and shrinkage. J. R. Stat. Soc. Ser. B. Stat. Methodol. 45 311–354.
  • Copas, J. B. (1997). Using regression models for prediction: Shrinkage and regression to the mean. Stat. Methods Med. Res. 6 167–183.
  • Cox, D. R. (1958). Two further applications of a model for binary regression. Biometrika 45 562–565.
  • Cui, W. and George, E. I. (2008). Empirical Bayes vs. fully Bayes variable selection. J. Statist. Plann. Inference 138 888–900.
  • Davidson, R. R. and Lever, W. E. (1970). The limiting distribution of the likelihood ratio statistic under a class of local alternatives. Sankhyā Ser. A 32 209–224.
  • Edwards, W., Lindman, H. and Savage, L. J. (1963). Bayesian statistical inference for psychological research. Psychological Review 70 193–242.
  • Fernández, C., Ley, E. and Steel, M. F. J. (2001). Benchmark priors for Bayesian model averaging. J. Econometrics 100 381–427.
  • Foster, D. P. and George, E. I. (1994). The risk inflation criterion for multiple regression. Ann. Statist. 22 1947–1975.
  • Geisser, S. (1984). On prior distributions for binary trials. Amer. Statist. 38 244–251.
  • George, E. I. and Foster, D. P. (2000). Calibration and empirical Bayes variable selection. Biometrika 87 731–747.
  • Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc. 102 359–378.
  • Goodman, S. N. (1999a). Toward evidence-based medical statistics. 1: The $P$-value fallacy. Annals of Internal Medicine 130 995–1004.
  • Goodman, S. N. (1999b). Toward evidence-based medical statistics. 2: The Bayes factor. Annals of Internal Medicine 130 1005–1013.
  • Gravestock, I. (2014). Bayesian tree models priors and posterior approximations. Master’s thesis, Univ. Zurich.
  • Held, L. (2010). A nomogram for $P$-values. BMC Medical Research Methodology 10 21.
  • Hjort, N. L. and Claeskens, G. (2003). Frequentist model average estimators. J. Amer. Statist. Assoc. 98 879–899.
  • Hu, J. and Johnson, V. E. (2009). Bayesian model selection using test statistics. J. R. Stat. Soc. Ser. B. Stat. Methodol. 71 143–158.
  • Johnson, V. E. (2005). Bayes factors based on test statistics. J. R. Stat. Soc. Ser. B. Stat. Methodol. 67 689–701.
  • Johnson, V. E. (2008). Properties of Bayes factors based on test statistics. Scand. J. Stat. 35 354–368.
  • Kass, R. E. and Wasserman, L. (1995). A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. J. Amer. Statist. Assoc. 90 928–934.
  • Lawless, J. F. and Singhal, K. (1978). Efficient screening of nonnormal regression models. Biometrics 34 318–327.
  • Lee, K. L., Woodlief, L. H., Topol, E. J., Weaver, W. D., Betriu, A., Col, J., Simoons, M., Aylward, P., Van de Werf, F. and Califf, R. M. (1995). Predictors of 30-day mortality in the era of reperfusion for acute myocardial infarction: Results from an international trial of 41,021 patients. Circulation 91 1659–1668.
  • Liang, F., Paulo, R., Molina, G., Clyde, M. A. and Berger, J. O. (2008). Mixtures of $g$ priors for Bayesian variable selection. J. Amer. Statist. Assoc. 103 410–423.
  • Lindley, D. V. (1957). A statistical paradox. Biometrika 44 187–192.
  • Malaguerra, A. (2012). Bayesian variable selection based on test statistics. Master’s thesis, Univ. Zurich.
  • Marra, G. and Wood, S. N. (2011). Practical variable selection for generalized additive models. Comput. Statist. Data Anal. 55 2372–2387.
  • Nelder, J. A. and Wedderburn, R. W. M. (1972). Generalized linear models. J. Roy. Statist. Soc. Ser. A 135 370–384.
  • Rue, H., Martino, S. and Chopin, N. (2009). Approximate Bayesian inference for latent Gaussian models using integrated nested Laplace approximations (with discussion). J. Roy. Statist. Soc. Ser. B 71 319–392.
  • Sabanés Bové, D. and Held, L. (2011a). Hyper-$g$ priors for generalized linear models. Bayesian Anal. 6 387–410.
  • Sabanés Bové, D. and Held, L. (2011b). Bayesian fractional polynomials. Stat. Comput. 21 309–324.
  • Sabanés Bové, D., Held, L. and Kauermann, G. (2014). Mixtures of $g$-priors for generalised additive model selection with penalised splines. J. Comput. Graph. Statist. DOI:10.1080/10618600.2014.912136.
  • Scott, J. G. and Berger, J. O. (2010). Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. Ann. Statist. 38 2587–2619.
  • Sellke, T., Bayarri, M. J. and Berger, J. O. (2001). Calibration of $p$-values for testing precise null hypotheses. Amer. Statist. 55 62–71.
  • Steyerberg, E. (2009). Clinical Prediction Models. Springer, New York.
  • van Houwelingen, J. C. and Le Cessie, S. (1990). Predictive value of statistical models. Stat. Med. 9 1303–1325.
  • Zellner, A. (1986). On assessing prior distributions and Bayesian regression analysis with $g$-prior distributions. In Bayesian Inference and Decision Techniques (P. K. Goel and A. Zellner, eds.). Stud. Bayesian Econometrics Statist. 6 233–243. North-Holland, Amsterdam.
  • Zellner, A. and Siow, A. (1980). Posterior odds ratios for selected regression hypotheses. In Bayesian Statistics: Proceedings of the First International Meeting Held in Valencia (J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 585–603. Univ. Valencia Press, Valencia.

The Institute of Mathematical Statistics

Statistical Science

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more