Bayesian Analysis

Bayesian Model Choice in Cumulative Link Ordinal Regression Models

Trevelyan J. McKinley, Michelle Morters, and James L. N. Wood

Full-text: Open access

Abstract

The use of the proportional odds (PO) model for ordinal regression is ubiquitous in the literature. If the assumption of parallel lines does not hold for the data, then an alternative is to specify a non-proportional odds (NPO) model, where the regression parameters are allowed to vary depending on the level of the response. However, it is often difficult to fit these models, and challenges regarding model choice and fitting are further compounded if there are a large number of explanatory variables. We make two contributions towards tackling these issues: firstly, we develop a Bayesian method for fitting these models, that ensures the stochastic ordering conditions hold for an arbitrary finite range of the explanatory variables, allowing NPO models to be fitted to any observed data set. Secondly, we use reversible-jump Markov chain Monte Carlo to allow the model to choose between PO and NPO structures for each explanatory variable, and show how variable selection can be incorporated. These methods can be adapted for any monotonic increasing link functions. We illustrate the utility of these approaches on novel data from a longitudinal study of individual-level risk factors affecting body condition score in a dog population in Zenzele, South Africa.

Article information

Source
Bayesian Anal., Volume 10, Number 1 (2015), 1-30.

Dates
First available in Project Euclid: 28 January 2015

Permanent link to this document
https://projecteuclid.org/euclid.ba/1422468421

Digital Object Identifier
doi:10.1214/14-BA884

Mathematical Reviews number (MathSciNet)
MR3420895

Zentralblatt MATH identifier
1334.62141

Keywords
Bayesian inference ordinal regression Markov chain Monte Carlo reversible-jump Bayesian model choice

Citation

McKinley, Trevelyan J.; Morters, Michelle; Wood, James L. N. Bayesian Model Choice in Cumulative Link Ordinal Regression Models. Bayesian Anal. 10 (2015), no. 1, 1--30. doi:10.1214/14-BA884. https://projecteuclid.org/euclid.ba/1422468421


Export citation

References

  • Agresti, A. (2010). Analysis of Ordinal Categorical Data. Wiley, 2nd edition.
  • Akaike, H. (1974). “A new look at statistical model identification.” IEEE Transactions on Automatic Control, AU-19: 195–223.
  • Albert, J. and Chib, S. (1997). “Bayesian methods for cumulative, sequential and two-step ordinal data regression models.” Technical report.
  • Albert, J. H. and Chib, S. (1993). “Bayesian analysis of binary and polychotomous response data.” Journal of the American Statistical Association, 88(422): 669–679.
  • Ananth, C. V. and Kleinbaum, D. G. (1997). “Regression models for ordinal responses: A review of methods and applications.” International Journal of Epidemiology, 26(6): 1323–1333.
  • Bender, R. and Grouven, U. (1998). “Using binary logistic regression models for ordinal data with non-proportional odds.” Journal of Clinical Epidemiology, 51(10): 809–816.
  • Brant, R. (1990). “Assessing proportionality in the proportional odds model for ordinal logistic regression.” Biometrics, 46(4): 1171–1178.
  • Brooks, S. P., Giudici, P., and Roberts, G. O. (2003). “Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions.” Journal of the Royal Statistical Society. Series B (Methodological), 65(1): 3–55.
  • Chib, S. (1995). “Marginal likelihood from the Gibbs output.” Journal of the American Statistical Association, 90(432): 1313–1321.
  • Chu, W. and Ghahramani, Z. (2005). “Gaussian processes for ordinal regression.” Journal of Machine Learning Research, 6: 1–48.
  • Cole, S. R., Allison, P. D., and Ananth, C. V. (2004). “Estimation of cumulative odds ratios.” Annals of Epidemiology, 14: 172–178.
  • Congdon, P. (2005). Bayesian Models for Categorical Data. Wiley.
  • Dellaportas, P., Forster, J. J., and Ntzoufras, I. (2002). “On Bayesian model and variable selection using MCMC.” Statistics and Computing, 12: 27–36.
  • Diggle, P. J., Heagerty, P., Liang, K.-Y., and Zeger, S. L. (2002). Analysis of Longitudinal Data. Oxford University Press, 2nd edition.
  • Fahrmeier, L. and Tutz, G. (1994). Multivariate Statistical Modelling Based on Generalized Linear Models. Springer.
  • Feinberg, S. E. (1980). The Analysis of Cross-Classified Categorical Data. Springer.
  • Gamerman, D. and Lopes, H. F. (2006). Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference. CRC Press, 2nd edition.
  • Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2004). Bayesian Data Analysis. Chapman and Hall/CRC, 2nd edition.
  • German, A. J. and Holden, S. L. (2006). “Subjective estimation of body condition can predict body fat mass as well as condition scoring with an established 9-point scale.” In BSAVA Congress 2006 Scientific Proceedings, 508. BSAVA Publications.
  • Gibbons, R. D. and Hedeker, D. (1997). “Random effects probit and logistic regression models for three-level data.” Biometrics, 53: 1527–1537.
  • Gilks, W., Richardson, S., and Spiegelhalter, D. (eds.) (1996). Markov Chain Monte Carlo In Practice. Chapman and Hall.
  • Green, P. J. (1995). “Reversible jump Markov chain Monte Carlo computation and Bayesian model determination.” Biometrika, 82(4): 711–732.
  • Haario, H., Saksman, E., and Tamminen, J. (2001). “An adaptive Metropolis algorithm.” Bernoulli, 7(2): 223–242.
  • Hartzel, J., Agresti, A., and Caffo, B. (2001). “Multinomial logit random effects models.” Statistical Modelling, 1: 81–102.
  • Hastie, D. I. and Green, P. J. (2012). “Model choice using reversible jump Markov chain Monte Carlo.” Statistica Neerlandica, 66(3): 309–338.
  • Hastings, W. (1970). “Monte Carlo sampling methods using Markov chains and their applications.” Biometrika, 57: 97–109.
  • Hedeker, D. (2003). “A mixed-effects multinomial logistic regression model.” Statistics in Medicine, 22: 1433–1446.
  • Hedeker, D. and Gibbons, R. D. (1994). “A random-effects ordinal regression model for multilevel analysis.” Biometrics, 50: 933–944.
  • Holmes, C. C. and Held, L. (2006). “Bayesian auxiliary variable models for binary and multinomial regression.” Bayesian Analysis, 1(1): 145–168.
  • Ishwaran, H. (2000). “Univariate and multirater ordinal cumulative link regression with covariate specific cutpoints.” The Canadian Journal of Statistics, 28(4): 715–730.
  • Ishwaran, H. and Gatsonis, C. A. (2000). “A general class of hierarchical ordinal regression models with applications to correlated ROC analysis.” The Canadian Journal of Statistics, 28(4): 731–750.
  • Jeffreys, H. (1935). “Some tests of significance, treated by the theory of probability.” Proceedings of the Cambridge Philosophy Society, 31: 203–222.
  • — (1961). The Theory of Probability. Oxford, 3rd edition.
  • Johnson, V. E. and Albert, J. H. (1999). Ordinal Data Modeling. Springer-Verlag, New York.
  • Kass, R. E. and Raftery, A. E. (1995). “Bayes Factors.” Journal of the American Statistical Association, 90(430): 773–795.
  • Krzanowski, W. J. (1998). An Introduction to Statistical Modelling. Arnold.
  • Lall, R., Campbell, M. J., Walters, S. J., Morgan, K., and MRC CFAS Co-operative (2002). “A review of ordinal regression models applied on health-related quality of life assessments.” Statistical Methods in Medical Research, 11: 49–67.
  • Lang, J. B. (1999). “Bayesian ordinal and binary regression models with a parametric family of mixture links.” Computational Statistics and Data Analysis, 31: 59–87.
  • Leon-Novelo, L. G., Zhou, X., Bekele, B. N., and Müller, P. (2010). “Assessing toxicities in a clinical trial: Bayesian inference for ordinal data nested within categories.” Biometrics, 66: 966–974.
  • Liang, H., Wu, H., and Zou, G. (2008). “A note on conditional AIC for linear mixed-effects models.” Biometrika, 95(3): 773–778.
  • Link, W. A. and Eaton, M. J. (2012). “On thinning of chains in MCMC.” Methods in Ecology and Evolution, 3: 112–115.
  • Liu, L. C. and Hedeker, D. (2006). “A mixed-effects regression model for longitudinal multivariate ordinal data.” Biometrics, 62: 261–268.
  • McCullagh, P. (1980). “Regression models for ordinal data (with discussion).” Journal of the Royal Statistical Society. Series B (Methodological), 42(2): 109–142.
  • Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., and Teller, E. (1953). “Equations of state calculations by fast computing machine.” Journal of Chemical Physics, 21: 1087–1091.
  • Morters, M., McKinley, T., Restif, O., Conlan, A., Cleaveland, S., Hampson, K., Whay, B., Damriyasa, I. M., and Wood, J. (2014). “The demography of free-roaming dog populations and applications to disease and population control.” to appear in Journal of Applied Ecology.
  • Mwalili, S. M., Lesaffre, E., and Declerck, D. (2005). “A Bayesian ordinal logistic regression model to correct for interobserver measurement error in a geographical oral health study.” Applied Statistics, 54: 77–93.
  • O’Brien, S. M. and Dunson, D. B. (2004). “Bayesian multivariate logistic regression.” Biometrics, 60: 739–746.
  • O’Hara, R. B. and Sillanpää, M. J. (2009). “A review of Bayesian variable selection methods: what, how and which.” Bayesian Analysis, 4(1): 85–118.
  • Paquet, U., Holden, S., and Naish-Guzman, A. (2005). “Bayesian hierarchical ordinal regression.” In Duch, W., Oja, E., and Zadrozny, S. (eds.), Artificial Neural Networks: Formal Models and Their Applications – ICANN 2005, 267–272. Springer-Verlag Berlin Heidelberg.
  • Peterson, B. and Harrell, F. E. (1990). “Partial proportional odds models for ordinal response variables.” Journal of the Royal Statistical Society. Series C (Applied Statistics), 39(2): 205–217.
  • Plummer, M., Best, N., Cowles, K., and Vines, K. (2006). “CODA: Convergence Diagnosis and Output Analysis for MCMC.” R News, 6(1): 7–11. URL http://CRAN.R-project.org/doc/Rnews/
  • R Core Team (2012). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-project.org/
  • Richardson, S. and Green, P. (1997). “On Bayesian analysis of mixtures with an unknown number of components (with discussion).” Journal of the Royal Statistical Society. Series B (Methodological), 59: 731–792.
  • Roberts, G. O. and Rosenthal, J. S. (2009). “Examples of adaptive MCMC.” Journal of Computational and Graphical Statistics, 18(2): 349–367.
  • Tutz, G. and Scholz, T. (2003). “Ordinal regression modelling between proportional odds and non-proportional odds.” Technical report, Institute of Statistics, University of Munich.
  • Urbanek, S. (2011). multicore: Parallel processing of R code on machines with multiple cores or CPUs. R package version 0.1-7. URL http://CRAN.R-project.org/package=multicore
  • Vaida, F. and Blanchard, S. (2005). “Conditional Akaike information for mixed-effects models.” Biometrika, 92(2): 351–370.
  • Venables, W. N. and Ripley, B. D. (2002). Modern Applied Statistics with S. New York: Springer, fourth edition. ISBN 0-387-95457-0. URL http://www.stats.ox.ac.uk/pub/MASS4
  • Viallefont, V., Raftery, A. E., and Richardson, S. (2001). “Variable selection and Bayesian model averaging in case-control studies.” Statistics in Medicine, 20: 3215–3230.
  • Waagepetersen, R. and Sorensen, D. (2001). “A tutorial on reversible jump MCMC with a view toward applications in QTL-mapping.” International Statistical Review, 69(1): 49–61.
  • Webb, E. L. and Forster, J. J. (2008). “Bayesian model determination for multivariate ordinal and binary data.” Computational Statistics and Data Analysis, 52: 2632–2649.
  • Yi, N., Banerjee, S., Pomp, D., and Yandell, B. S. (2007). “Bayesian mapping of genomewide interacting quantitative trait loci for ordinal traits.” Genetics, 176: 1855–1864.

Supplemental materials

  • Supplementary material: Supplementary Materials: Bayesian model choice in cumulative link ordinal regression models: an application in a longitudinal study of risk factors affecting body condition score in a dog population in Zenzele, South Africa.