Bayesian Analysis

Bayes Factor Testing of Multiple Intraclass Correlations

Joris Mulder and Jean-Paul Fox

Full-text: Open access

Enhanced PDF (569 KB)

Abstract

The intraclass correlation plays a central role in modeling hierarchically structured data, such as educational data, panel data, or group-randomized trial data. It represents relevant information concerning the between-group and within-group variation. Methods for Bayesian hypothesis tests concerning the intraclass correlation are proposed to improve decision making in hierarchical data analysis and to assess the grouping effect across different group categories. Estimation and testing methods for the intraclass correlation coefficient are proposed under a marginal modeling framework where the random effects are integrated out. A class of stretched beta priors is proposed on the intraclass correlations, which is equivalent to shifted F priors for the between groups variances. Through a parameter expansion it is shown that this prior is conditionally conjugate under the marginal model yielding efficient posterior computation. A special improper case results in accurate coverage rates of the credible intervals even for minimal sample size and when the true intraclass correlation equals zero. Bayes factor tests are proposed for testing multiple precise and order hypotheses on intraclass correlations. These tests can be used when prior information about the intraclass correlations is available or absent. For the noninformative case, a generalized fractional Bayes approach is developed. The method enables testing the presence and strength of grouped data structures without introducing random effects. The methodology is applied to a large-scale survey study on international mathematics achievement at fourth grade to test the heterogeneity in the clustering of students in schools across countries and assessment cycles.

Article information

Source
Bayesian Anal., Volume 14, Number 2 (2019), 521-552.

Dates
First available in Project Euclid: 10 August 2018

Permanent link to this document
https://projecteuclid.org/euclid.ba/1533866668

Digital Object Identifier
doi:10.1214/18-BA1115

Mathematical Reviews number (MathSciNet)
MR3934096

Zentralblatt MATH identifier
07045441

Keywords
Intraclass correlations Bayes factors stretched beta priors shifted F priors hierarchical models

Rights
Creative Commons Attribution 4.0 International License.

Citation

Mulder, Joris; Fox, Jean-Paul. Bayes Factor Testing of Multiple Intraclass Correlations. Bayesian Anal. 14 (2019), no. 2, 521--552. doi:10.1214/18-BA1115. https://projecteuclid.org/euclid.ba/1533866668


Export citation

References

  • Albert, J. H. and Chib, S. (1993). “Bayesian Analysis of Binary and Polychotomous Response Data.” Journal of the American Statistical Association, 88(422): 669–679. URL http://www.tandfonline.com/doi/abs/10.1080/01621459.1993.10476321
    Zentralblatt MATH: 0774.62031
    Digital Object Identifier: doi:10.1080/01621459.1993.10476321
  • Armagan, A., Dunson, D. B., and Clyde, M. (2011). “Generalized Beta Mixtures of Gaussians.” In Shawe-Taylor, J., Zemel, R., Bartlett, P., Pereira, F., and Weinberger, K. (eds.), Advances in Neural Information Processing Systems 24, 523–531.
  • Bayarri, M. J., Berger, J. O., Forte, A., and García-Donato, G. (2012). “Criteria for Bayesian model choice with application to variable selection.” The Annals of Statistics, 40: 1550–1577.
    Zentralblatt MATH: 1257.62023
    Digital Object Identifier: doi:10.1214/12-AOS1013
    Project Euclid: euclid.aos/1346850065
  • Berger, J. O. (1994). “An overview of robust Bayesian analysis (with discussion).” Test, 3: 5–124.
    Zentralblatt MATH: 0827.62026
    Digital Object Identifier: doi:10.1007/BF02562676
  • Berger, J. O. (2006). “The case for objective Bbayesian analysis.” Bayesian Analysis, 1: 385–402.
    Zentralblatt MATH: 1331.62042
    Digital Object Identifier: doi:10.1214/06-BA115
  • Berger, J. O. and Bernardo, J. (1992). Reference priors in a variance component’s problem, 323–340. London: Oxford University Press.
  • Berger, J. O., De Oliveira, V., and Sansó, B. (2006). “Objective Bayesian analysis of spatially correlated data.” Journal of the American Statistical Association, 96: 1361–1374.
    Zentralblatt MATH: 1051.62095
    Digital Object Identifier: doi:10.1198/016214501753382282
  • Berger, J. O., Liseo, B., and Wolpert, R. L. (1999). “Integrated likelihood methods for eliminating nuisance parameters.” Statistical Science, 14(1): 1–22. URL http://www.jstor.org/stable/2676641
    Zentralblatt MATH: 1059.62521
    Digital Object Identifier: doi:10.1214/ss/1009212518
    Project Euclid: euclid.ss/1009211804
  • Berger, J. O. and Mortera, J. (1995). “Discussion to fractional Bayes factors for model comparison (by O’Hagan).” Journal of the Royal Statistical Society Series B, 56: 130.
  • Berger, J. O. and Pericchi, L. (2001). “Objective Bayesian methods for model selection: Introduction and comparison (with discussion).” In Lahiri, P. (ed.), Model Selection, volume 38 of Monograph Series, 135–207. Beachwood Ohio, institute of mathematical statistics lecture notes edition.
  • Berger, J. O. and Pericchi, L. R. (1996). “The intrinsic Bayes factor for model selection and prediction.” Journal of the American Statistical Association, 91: 109–122.
    Zentralblatt MATH: 0870.62021
    Digital Object Identifier: doi:10.1080/01621459.1996.10476668
  • Berger, J. O. and Sun, D. (2008). “Objective priors for the bivariate normal model.” The Annals of Statistics, 36: 963–982.
    Zentralblatt MATH: 1133.62014
    Digital Object Identifier: doi:10.1214/07-AOS501
    Project Euclid: euclid.aos/1205420525
  • Böing-Messing, F., van Assen, M. A. L. M., Hofman, A., Hoijtink, H., and Mulder, J. “Bayesian evaluation of constrained hypotheses on variances of multiple independent groups.” Psychological Methods, 22: 262–287.
  • Box, G. E. P. and Tiao, G. C. (1973). Bayesian Inference in Statistical Snalysis. Reading, MA: Addison-Wesley.
    Zentralblatt MATH: 0271.62044
  • Braeken, J., Mulder, J., and Wood, S. (2015). “Relative effects at work: Bayes factors for order hypotheses.” Journal of Management, 41: 544–573.
  • Chung, Y. and Dey, D. K. (1998). “Bayesian approach to estimation of intraclass correlation using reference prior.” Communications in Statistics, 26: 2241–2255.
    Zentralblatt MATH: 0917.62019
    Digital Object Identifier: doi:10.1080/03610929808832225
  • Conigliani, C. and O’Hagan, A. (2000). “Sensitivity of the fractional Bayes factor to prior distributions.” The Canadian Journal of Statistics, 28: 343–352.
    Zentralblatt MATH: 0962.62022
    Digital Object Identifier: doi:10.2307/3315983
  • Consonni, G. and Paroli, R. (2017). “Objective Bayesian Comparison of Constrained Analysis of Variance Models.” Psychometrika, 82: 589–609.
    Zentralblatt MATH: 1402.62040
    Digital Object Identifier: doi:10.1007/s11336-016-9516-y
  • De Santis, F. and Spezzaferri, F. (2001). “Consistent fractional Bayes factor for nested normal linear models.” Journal of Statistical Planning and Inference, 97: 305–321.
    Zentralblatt MATH: 1015.62021
    Digital Object Identifier: doi:10.1016/S0378-3758(00)00240-8
  • Donner, A. and Zou, G. (2002). “Testing the equality of dependent intraclass correlation coefficients.” Journal of the Royal Statistical Society Series D, 51: 379–379.
  • Fox, J.-P. (2010). Bayesian Item Response Modeling: Theory and Applications. New York: Springer.
    Zentralblatt MATH: 1271.62012
  • Fox, J.-P., Mulder, J., and Sinharay, S. (2017). “Bayes factor covariance testing in item response models.” Psychometrika, 82: 979–1006.
    Zentralblatt MATH: 1402.62308
    Digital Object Identifier: doi:10.1007/s11336-017-9577-6
  • García-Donato, G. and Sun, D. (2007). “Objective priors for hypothesis testing in one-way random effects models.” Canadian Journal of Statistics, 35: 302–320.
  • Gelman, A. (2006). “Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper).” Bayesian Analysis, 1: 515–534.
    Zentralblatt MATH: 1331.62139
    Digital Object Identifier: doi:10.1214/06-BA117A
  • Ghosh, J. K. and Mukerjee, R. (1992). Non-informative priors, 195–210. Oxford University Press.
  • Gilks, W. R. (1995). “Discussion to fractional Bayes factors for model comparison (by O’Hagan).” Journal of the Royal Statistical Society Series B, 56: 118–120.
  • Griffin, J. and Brown, P. (2005). “Alternative prior distributions for variable selection with very many more variables than observations.” Technical report, University of Warwick.
  • Gu, X., Mulder, J., and Hoijtink, H. (2014). “Bayesian evaluation of inequality constrained hypotheses.” Psychological Methods, 9(4): 511-527.
  • Gu, X., Mulder, J., Decovíc, M., and Hoijtink, H. (2017). “Approximated adjusted fractional Bayes factors: A general method for testing informative hypotheses.” British Journal of Mathematical and Statistical Psychology.
  • Haldane, J. B. S. (1932). “A note on inverse probability.” Mathematical Proceedings of the Cambridge Philosophical Society, 28: 55–61.
    Zentralblatt MATH: 0003.35602
    Digital Object Identifier: doi:10.1017/S0305004100010495
  • Harris, J. A. (1913). “On the calculation of intra-class and inter-class coefficients of correlation from class moments when the number of possible combinations is large.” Biometrika, 9: 446–472.
  • Hedges, L. V. and Hedberg, E. C. (2007). “Intraclass Correlation Values for Planning Group-Randomized Trials in Education.” Educational Evaluation and Policy Analysis, 29(1): 60–87. URL http://www.jstor.org/stable/30128045
  • Hoijtink, H. (2011). Informative Hypotheses: Theory and Practice for Behavioral and Social Scientists. New York: Chapman & Hall/CRC.
  • Hoijtink, H., Gu, X., and Mulder, J. (2018). “Bayesian evaluation of informative hypotheses for multiple populations.” British Journal of Mathematical and Statistical Psychology.
    Zentralblatt MATH: 06872266
    Digital Object Identifier: doi:10.1111/bmsp.12110
  • Jeffreys, H. (1961). Theory of Probability-3rd ed. New York: Oxford University Press.
    Zentralblatt MATH: 0116.34904
  • Johnson, V. E. and Rossell, D. (2010). “On the use of non-local prior densities in Bayesian hypothesis tests.” Journal of the Royal Statistical Society Series B, 72: 143–170.
    Mathematical Reviews (MathSciNet): MR2830762
    Digital Object Identifier: doi:10.1111/j.1467-9868.2009.00730.x
  • Kass, R. E. and Vaidyanathan, S. K. (1992). “Approximate Bayes Factors and Orthogonal Parameters, with Application to Testing Equality of Two Binomial Proportions.” Journal of the Royal Statistical Society, Series B, 54: 129–144.
    Zentralblatt MATH: 0777.62032
    Digital Object Identifier: doi:10.1111/j.2517-6161.1992.tb01868.x
  • Kass, R. E. and Wasserman, L. (1995). “A Reference Bayesian Test for Nested Hypotheses and Its Relationship to the Schwarz Criterion.” Journal of the American Statistical Association, 90: 928–934.
    Zentralblatt MATH: 0851.62020
    Digital Object Identifier: doi:10.1080/01621459.1995.10476592
  • Kinney, S. and Dunson, D. B. (2007). “Fixed and Random Effects Selection in Linear and Logistic Models.” Biometrics, 63: 690–698.
    Zentralblatt MATH: 1147.62022
    Digital Object Identifier: doi:10.1111/j.1541-0420.2007.00771.x
  • Kish, L. (1965). Survey sampling. Chichester: Wiley.
    Zentralblatt MATH: 0151.23403
  • Konishi, S. and Gupta, K. A. (1989). “Testing the equality of several intraclass correlation coefficients.” Journal of Statistical Planning and Inference, (1): 93–105.
    Zentralblatt MATH: 0666.62057
    Digital Object Identifier: doi:10.1016/0378-3758(89)90022-0
  • Lancaster, H. O. (1965). “The Helmert matrices.” The American Mathematical Monthly, 72: 4–12.
    Zentralblatt MATH: 0124.01102
    Digital Object Identifier: doi:10.1080/00029890.1965.11970483
  • Liang, F., Paulo, R., Molina, G., Clyde, M. A., and Berger, J. O. (2008). “Mixtures of $g$ priors for Bayesian variable selection.” Journal of American Statistical Association, 103(481): 410–423.
    Zentralblatt MATH: 1335.62026
    Digital Object Identifier: doi:10.1198/016214507000001337
  • Liang, K.-Y. and Zeger, S. L. (1986). “Longitudinal data analysis using generalized linear models.” Biometrika, 73(1): 13.
    Zentralblatt MATH: 0595.62110
    Digital Object Identifier: doi:10.1093/biomet/73.1.13
  • Ly, A., Verhagen, J., and Wagenmakers, E.-J. (2016). “An evaluation of alternative methods for testing hypotheses, from the perspective of Harold Jeffreys.” Journal of Mathematical Psychology, 72: 43–55.
    Zentralblatt MATH: 1357.62118
    Digital Object Identifier: doi:10.1016/j.jmp.2016.01.003
  • Mulder, J. (2014). “Prior adjusted default Bayes factors for testing (in)equality constrained hypotheses.” Computational Statistics and Data Analysis, 71: 448–463.
    Zentralblatt MATH: 06975400
    Digital Object Identifier: doi:10.1016/j.csda.2013.07.017
  • Mulder, J. (2016). “Bayes factors for testing order-constrained hypotheses on correlations.” Journal of Mathematical Psychology, 72: 104–115.
    Zentralblatt MATH: 1357.62122
    Digital Object Identifier: doi:10.1016/j.jmp.2014.09.004
  • Mulder, J. and Fox, J.-P. (2013). “Bayesian tests on components of the compound symmetry covariance matrix.” Statistics and Computing, 23: 109–122.
    Zentralblatt MATH: 1322.62111
    Digital Object Identifier: doi:10.1007/s11222-011-9295-3
  • Mulder, J. and Fox, J.-P. (2018). “Supplemetary material for “Bayes factor testing of multiple intraclass correlations”.” Bayesian Analysis.
  • Mulder, J., Klugkist, I., van de Schoot, A., Meeus, W., Selfhout, M., and Hoijtink, H. (2009). “Bayesian Model Selection of Informative Hypotheses for Repeated Measurements.” Journal of Mathematical Psychology, 53: 530–546.
    Zentralblatt MATH: 1181.62026
    Digital Object Identifier: doi:10.1016/j.jmp.2009.09.003
  • Mulder, J. and Pericchi, L. R. (2018). “The matrix-$F$ prior for estimating and testing covariance matrices.” Bayesian Analysis.
    Zentralblatt MATH: 06989981
    Digital Object Identifier: doi:10.1214/17-BA1092
  • Mullis, I., Martin, M. O., Foy, P., and Arora, A. (2011). TIMSS 2011 International Results in Mathematics. TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College Chestnut Hill, MA, USA and International Association for the Evaluation of Educational Achievement (IEA) IEA Secretariat, Amsterdam, the Netherlands, first edition.
  • O’Hagan, A. (1995). “Fractional Bayes Factors for Model Comparison (with discussion).” Journal of the Royal Statistical Society Series B, 57: 99–138.
    Mathematical Reviews (MathSciNet): MR1325379
    Digital Object Identifier: doi:10.1111/j.2517-6161.1995.tb02017.x
  • O’Hagan, A. (1997). “Properties of intrinsic and fractional Bayes factors.” Test, 6: 101–118.
    Zentralblatt MATH: 0891.62001
    Digital Object Identifier: doi:10.1007/BF02564428
  • Olson, J. F., Martin, M. O., and Mullis, I. V. S. (2008). TIMSS 2007 Technical Report. Chestnut Hill, MA: TIMSS and PIRLS International Study Center, Boston College, first edition.
  • Pauler, D. K., Wakefield, J. C., and Kass, R. E. (1999). “Bayes Factors and Approximations for Variance Component Models.” Journal of the American Statistical Association, 94(448): 1242–1253.
    Zentralblatt MATH: 0998.62017
    Digital Object Identifier: doi:10.1080/01621459.1999.10473877
  • Pérez, M. E., Pericchi, L. R., and Ramirez, I. C. (2017). “The Scaled Beta2 Distribution as a Robust Prior for Scales.” Bayesian Analysis, 12: 615.
    Zentralblatt MATH: 1384.62048
    Digital Object Identifier: doi:10.1214/16-BA1015
  • Pericchi, L. R. (2005). “Model selection and hypothesis testing based on objective probabilities and Bayes Factors.” In D. K. Dey, C. R. Rao (eds.), Bayesian Thinking: Modeling and Computation, 115–149. Elsevier, Amsterdam, The Netherlands.
  • Pericchi, L. R., Liu, G., and Torres, D. (2008). “Objective Bayes factors for informative hypotheses: “Completing” the informative hypothesis and “splitting” the Bayes factors.” In H. Hoijtink, I. Klugkist, and P. Boelen (eds.), Bayesian Evaluation of Informative Hypotheses, 131–154. New York: Springer.
  • Polson, N. G. and Scott, J. G. (2012). “On the Half-Cauchy Prior for a Global Scale Parameter.” Bayesian Analysis, 7.
    Zentralblatt MATH: 1330.62148
    Digital Object Identifier: doi:10.1214/12-BA730
  • Raftery, A. E. (1995). “Bayesian model selection in social research.” Sociological Methodology, 25: 111–163.
  • Raudenbush, S. W. (1997). “Statistical analysis and optimal design for cluster randomized trials.” Psychological Methods, (2): 173–185. URL http://dx.doi.org/10.1037/1082-989X.2.2.173
  • Schwarz, G. E. (1978). “Estimating the dimension of a model.” Annals of Statistics, 6: 461–464.
    Zentralblatt MATH: 0379.62005
    Digital Object Identifier: doi:10.1214/aos/1176344136
    Project Euclid: euclid.aos/1176344136
  • Scott, J. G. and Berger, J. O. (2006). “An exploration of aspects of Bayesian multiple testing.” Journal of Statistical Planning and Inference, 136: 2144–2162.
    Zentralblatt MATH: 1087.62039
    Digital Object Identifier: doi:10.1016/j.jspi.2005.08.031
  • Searle, S. R., Casella, G., and McCulloch, C. E. (1992). Variance Components. John Wiley & Sons, Inc., Hoboken, NJ, USA, first edition.
    Zentralblatt MATH: 1108.62064
  • Sellke, T., Bayarri, M. J., and Berger, J. O. (2001). “Calibration of $p$ Values for Testing Precise Null Hypotheses.” The American Statistician, 55: 62–71.
    Zentralblatt MATH: 1182.62053
    Digital Object Identifier: doi:10.1198/000313001300339950
  • Severini, T. A., Mukerjee, R., and Ghosh, M. (2002). “On an exact probability matching property of right-invariant priors.” Biometrika, 89: 952–957.
    Zentralblatt MATH: 1034.62022
    Digital Object Identifier: doi:10.1093/biomet/89.4.952
  • Silvapulle, M. J. and Sen, P. K. (2004). Constrained Statistical Inference: Inequality, Order, and Shape Restrictions. Hoboken, NJ: John Wiley, second edition.
    Zentralblatt MATH: 1077.62019
  • Spiegelhalter, D. J. (2001). “Bayesian methods for cluster randomized trials with continuous responses.” Statistics in Medicine, 20(3): 435–452. URL http://dx.doi.org/10.1002/1097-0258(20010215)20:3<435::AID-SIM804>3.0.CO;2-E
  • Stein, C. (1985). On the coverage probability of confidence sets based on a prior distribution, 485–514. PWN-Polish Scientific Publishers.
  • Van Geel, M., Keuning, T., Visscher, A., and Fox, J.-P. (2017). “Changes in educators’ data literacy during a data-based decision making intervention.” Teaching and Teacher Education, 64: 187–198.
  • Welch, B. and Peers, H. W. (1963). “On formulae for confidence points based on integrals of weighted likelihoods.” Journal of the Royal Statistical Society Series B, 25: 318–29.
    Zentralblatt MATH: 0117.14205
    Digital Object Identifier: doi:10.1111/j.2517-6161.1963.tb00512.x
  • Westfall, P. and Gönen, M. (1996). “Asymptotic properties of ANOVA Bayes factors.” Communications in Statistics: Theory and Methods, 25: 3101–3123.
    Zentralblatt MATH: 0887.62032
    Digital Object Identifier: doi:10.1080/03610929608831888
  • Ye, K. (1994). “Bayesian reference prior analysis on the ratio of variances for the balanced one-way random effect model.” Journal of Statistical Planning and Inference, 41: 267–280.
    Zentralblatt MATH: 0816.62029
    Digital Object Identifier: doi:10.1016/0378-3758(94)90023-X
  • Zellner, A. (1986). “On assessing prior distributions and Bayesian regression analysis with $g$-prior distributions.” In Goel, P. K. and Zellner, A. (eds.), Bayesian inference and decision techniques: Essays in honor of Bruno de Finetti, 233–243. Amsterdam: Elsevier.
  • Zellner, A. and Siow, A. (1980). “Posterior odds ratios for selected regression hypotheses.” In J. M. Bernardo, D. V. Lindley, A. F. M. Smith, M. H. DeGroot (ed.), Bayesian statistics: proceedings of the first international meeting held in Valencia, 585–603. Spain: University of Valencia.
    Mathematical Reviews (MathSciNet): MR638871
    Zentralblatt MATH: 0457.62004

Supplemental materials

  • The supplementary material for “Bayes Factor Testing of Multiple Intraclass Correlations”. The supplementary material for “Bayes factor testing of multiple intraclass correlations” contains the proof of Lemma 1, the proof of Lemma 2, the conditional posterior distributions for the Gibbs sampler, the proof of Lemma 3, the Gibbs sampler under a constrained model, the analytic expression of the marginal likelihood (with derivation) for a standard random intercept model using fractional Bayes methodology, and a simulation study when testing interval hypotheses.
    Digital Object Identifier: doi:10.1214/18-BA1115SUPP
    Supplemental PDF (593 KB)

International Society for Bayesian Analysis

Bayesian Analysis

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more