Bayesian Analysis

Mixtures of $g$-Priors for Analysis of Variance Models with a Diverging Number of Parameters

Min Wang

Abstract

We consider Bayesian approaches for the hypothesis testing problem in the analysis-of-variance (ANOVA) models. With the aid of the singular value decomposition of the centered designed matrix, we reparameterize the ANOVA models with linear constraints for uniqueness into a standard linear regression model without any constraint. We derive the Bayes factors based on mixtures of $g$-priors and study their consistency properties with a growing number of parameters. It is shown that two commonly used hyper-priors on $g$ (the Zellner-Siow prior and the beta-prime prior) yield inconsistent Bayes factors due to the presence of an inconsistency region around the null model. We propose a new class of hyper-priors to avoid this inconsistency problem. Simulation studies on the two-way ANOVA models are conducted to compare the performance of the proposed procedures with that of some existing ones in the literature.

Article information

Source
Bayesian Anal., Volume 12, Number 2 (2017), 511-532.

Dates
First available in Project Euclid: 5 July 2016

https://projecteuclid.org/euclid.ba/1467722664

Digital Object Identifier
doi:10.1214/16-BA1011

Mathematical Reviews number (MathSciNet)
MR3620743

Citation

Wang, Min. Mixtures of $g$ -Priors for Analysis of Variance Models with a Diverging Number of Parameters. Bayesian Anal. 12 (2017), no. 2, 511--532. doi:10.1214/16-BA1011. https://projecteuclid.org/euclid.ba/1467722664

References

• Bayarri, M. J., Berger, J. O., Forte, A., and García-Donato, G. (2012). “Criteria for Bayesian model choice with application to variable selection.” Annals of Statistics, 40(3): 1550–1577.
• Bayarri, M. J. and García-Donato, G. (2007). “Extending conventional priors for testing general hypotheses in linear models.” Biometrika, 94(1): 135–152.
• Becker, M. and Klößner, S. (2013). PearsonDS: Pearson Distribution System. R package version 0.97. URL http://CRAN.R-project.org/package=PearsonDS
• Berger, J. O., Ghosh, J. K., and Mukhopadhyay, N. (2003). “Approximations and consistency of Bayes factors as model dimension grows.” Journal of Statistical Planning and Inference, 112(1-2): 241–258.
• Berger, J. O. and Pericchi, L. R. (1996). “The intrinsic Bayes factor for model selection and prediction.” Journal of the American Statistical Association, 91(433): 109–122.
• Berger, J. O. and Pericchi, L. R. (2001). “Objective Bayesian methods for model selection: introduction and comparison.” In Model selection, volume 38 of IMS Lecture Notes Monogr. Ser., 135–207. Beachwood, OH: Inst. Math. Statist. With discussion by J. K. Ghosh, Tapas Samanta, and Fulvio De Santis, and a rejoinder by the authors.
• Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010). “The horseshoe estimator for sparse signals.” Biometrika, 97(2): 465–480.
• Draper, N. and Smith, H. (1998). Applied Regression Analysis. New York: Wiley-Interscience.
• Fernández, C., Ley, E., and Steel, M. F. J. (2001). “Benchmark priors for Bayesian model averaging.” Journal of Econometrics, 100(2): 381–427.
• Fujikoshi, Y. (1993). “Two-way ANOVA models with unbalanced data.” Discrete Mathematics, 116(1-3): 315–334.
• George, E. I. and Foster, D. P. (2000). “Calibration and empirical Bayes variable selection.” Biometrika, 87(4): 731–747.
• Girón, F. J., Moreno, E., Casella, G., and Martínez, M. L. (2010). “Consistency of objective Bayes factors for nonnested linear models and increasing model dimension.” Revista de la Real Academia de Ciencias Exactas, Físicas Y Naturales. Serie A, Matemáticas (Ed. Impresa), 104(1): 57–67.
• Guo, R. and Speckman, P. L. (2009). “Bayes factor consistency in linear models.” In the 2009 International Workshop on Objective Bayes Methodology, Philadelphia, June 5-9, 2009.
• Jeffreys, H. (1961). Theory of Probability. Statistics and Computing, 3rd edn. London: Oxford University Press.
• Johnson, V. E. and Rossell, D. (2012). “Bayesian model selection in high-dimensional settings.” Journal of the American Statistical Association, 107(498): 649–660.
• Kass, R. E. and Raftery, A. E. (1995). “Bayes factors.” Journal of the American Statistical Association, 90(430): 773–795.
• Lazic, S. (2008). “Why we should use simpler models if the data allow this: relevance for ANOVA designs in experimental biology.” BMC Physiology, 8(1): 1–7.
• Ley, E. and Steel, M. F. J. (2012). “Mixtures of $g$-priors for Bayesian model averaging with economic applications.” Journal of Econometrics, 171(2): 251–266.
• Liang, F., Paulo, R., Molina, G., Clyde, M. A., and Berger, J. O. (2008). “Mixtures of $g$ priors for Bayesian variable selection.” Journal of the American Statistical Association, 103(481): 410–423.
• Maruyama, Y. (2012). “A Bayes factor with reasonable model selection consistency for ANOVA model.” ArXiv:0906.4329v2 [stat.ME].
• Maruyama, Y. and George, E. I. (2011). “Fully Bayes factors with a generalized $g$-prior.” Annals of Statistics, 39(5): 2740–2765.
• Min, X. and Sun, D. (2015). “Bayesian model selection for a linear model with grouped covariates.” Annals of the Institute of Statistical Mathematics, 1–27.
• Moreno, E., Girón, J., and Casella, G. (2015). “Posterior model consistency in variable selection as the model dimension grows.” Statistical Science, 30(2): 228–241.
• Moreno, E., Girón, F. J., and Casella, G. (2010). “Consistency of objective Bayes factors as the model dimension grows.” Annals of Statistics, 38(4): 1937–1952.
• Pearson, K. (1895). “Contributions to the Mathematical Theory of Evolution. II. Skew Variation in Homogeneous Material.” 186: 343–414.
• Qian, S. and Shen, Z. (2007). “Ecological applications of multilevel analysis of variance.” Ecology, 88(10): 2489–2495.
• R Development Core Team (2011). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. URL http://www.R-project.org/
• Rouder, J. N., Morey, R. D., Speckman, P. L., and Province, J. M. (2012). “Default Bayes factors for ANOVA designs.” Journal of Mathematical Psychology, 56(5): 356–374.
• Searle, S., Casella, G., and McCulloch, C. (1992). Variance Components. Wiley, New York.
• VanLeeuwen, D. (1997). “Assessing reliability of measurements with generalizability theory: an application to inter-rater reliability.” Journal of Agricultural Education, 38(3): 36–42.
• Wang, M. (2016). “Supplementary material for “Mixtures of $g$-priors for analysis of variance models with a diverging number of parameters”” Bayesian Analysis.
• Wang, M. and Sun, X. (2013). “Bayes factor consistency for unbalanced ANOVA models.” Statistics, 47(5): 1104–1115.
• Wang, M. and Sun, X. (2014). “Bayes factor consistency for nested linear models with a growing number of parameters.” Journal of Statistical Planning and Inference, 147: 95–105.
• Wetzels, R., Grasman, R. P. P. P., and Wagenmakers, E.-J. (2012). “A default Bayesian hypothesis test for ANOVA designs.” The American Statistician, 66(2): 104–111.
• Womack, A. J., León-Novelo, L., and Casella, G. (2014). “Inference from intrinsic Bayes’ procedures under model selection and uncertainty.” Journal of the American Statistical Association, 109(507): 1040–1053.
• Zellner, A. (1986). “On assessing prior distributions and Bayesian regression analysis with $g$-prior distributions.” In Bayesian inference and decision techniques, volume 6 of Stud. Bayesian Econometrics Statist., 233–243. North-Holland, Amsterdam.
• Zellner, A. and Siow, A. (1980). “Posterior odds ratios for selected regression hypotheses.” Trabajos de Estadística Y de Investigación Operativa, 31(1): 585–603.