The Annals of Applied Statistics

Nonparametric inference procedure for percentiles of the random effects distribution in meta-analysis

Rui Wang, Lu Tian, Tianxi Cai, and L. J. Wei

Full-text: Open access


To investigate whether treating cancer patients with erythropoiesis-stimulating agents (ESAs) would increase the mortality risk, Bennett et al. [Journal of the American Medical Association 299 (2008) 914–924] conducted a meta-analysis with the data from 52 phase III trials comparing ESAs with placebo or standard of care. With a standard parametric random effects modeling approach, the study concluded that ESA administration was significantly associated with increased average mortality risk. In this article we present a simple nonparametric inference procedure for the distribution of the random effects. We re-analyzed the ESA mortality data with the new method. Our results about the center of the random effects distribution were markedly different from those reported by Bennett et al. Moreover, our procedure, which estimates the distribution of the random effects, as opposed to just a simple population average, suggests that the ESA may be beneficial to mortality for approximately a quarter of the study populations. This new meta-analysis technique can be implemented with study-level summary statistics. In contrast to existing methods for parametric random effects models, the validity of our proposal does not require the number of studies involved to be large. From the results of an extensive numerical study, we find that the new procedure performs well even with moderate individual study sample sizes.

Article information

Ann. Appl. Stat., Volume 4, Number 1 (2010), 520-532.

First available in Project Euclid: 11 May 2010

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Bivariate beta conditional permutation test erythropoiesis-stimulating agents logit-normal two-level hierachical model


Wang, Rui; Tian, Lu; Cai, Tianxi; Wei, L. J. Nonparametric inference procedure for percentiles of the random effects distribution in meta-analysis. Ann. Appl. Stat. 4 (2010), no. 1, 520--532. doi:10.1214/09-AOAS280.

Export citation


  • Bennett, C. L., Silver, S. M., Djulbegovic, B., Samaras, A. T., Blau, C. A., Gleason, K. J., Barnato, S. E., Elverman, K. M., Courtney, D. M., McKoy, J. M., Edwards, B. J., Tigue, C. C., Raisch, D. W., Yarnold, P. R., Dorr, D. A., Kuzel, T. M., Tallman, M. S., Trifilio, S. M., West, D. P., Lai, S. Y. and Henke, M. (2008). Venous thromboembolism and mortality associated with recombinant Erythropoietin and Darbepoetin administration for the treatment of cancer-associated anemia. Journal of the American Medical Association 299 914–924.
  • Biggerstaff, B. J. and Tweedie, R. L. (1997). Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. Statistics in Medicine 16 753–768.
  • Bohning, D., Malzahn, U., Dietz, E. and Schlattmann, P. (2002). Some general points in estimating heterogeneity variance with the DerSimonian–Laird estimator. Biostatistics 3 445–457.
  • Brockwell, S. E. and Gordon, I. R. (2001). A comparison of statistical methods for meta-analysis. Statistics in Medicine 20 825–840.
  • Brockwell, S. E. and Gordon, I. R. (2007). A simple method for inference on an overall effect in meta-analysis. Statistics in Medicine 26 4531–4543.
  • Cai, T., Cheng, S. C. and Wei, L. J. (2002). Semiparametric mixed-effects models for clustered failure time data. J. Amer. Statist. Assoc. 97 514–522.
  • DerSimonian, R. and Laird, N. M. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials 7 177–188.
  • DerSimonian, R. and Kacker, R. (2007). Random-effects model for meta-analysis of clinical trials: An update. Contemporary Clinical Trials 28 105–114.
  • Emerson, J. D., Hoaglin, D. C. and Mosteller, F. (1993). A modified random-effect procedure for combining risk difference in sets of 2×2 tables from clinical trials. Journal of the Italian Statistical Society 2 269–290.
  • Hardy, R. J. and Thompson, S. G. (1996). A likelihood approach to meta-analysis with random effects. Statistics in Medicine 15 619–629.
  • Hartung, J. (1999). An alternative method for meta-analysis. Biometrical Journal 41 901–916.
  • Hartung, J. and Knapp, G. (2001a). A refined method for the meta-analysis of controlled clinical trials with binary outcome. Statistics in Medicine 20 3875–3889.
  • Hartung, J. and Knapp, G. (2001b). On tests of the overall treatment effect in meta-analysis with normally distributed responses. Statistics in Medicine 20 1771–1782.
  • Henderson, R., Diggle, P. and Dobson, A. (2000). Joint modelling of longitudinal measurements and event time data. Biostatistics 1 465–480.
  • Hogan, J. W. and Laird, N. M. (1997). Mixture models for the joint distribution of repeated measures and event times. Statistics in Medicine 16 239–257.
  • Hougaard, P. (1995). Frailty models for survival data. Lifetime Data Anal. 1 255–273.
  • Laird, N. M. and Ware, J. H. (1982). Random effects models for longitudinal data. Biometrics 38 963–974.
  • Lam, K. F., Lee, Y. W. and Leung, T. L. (2002). Modeling multivariate survival data by a semiparametric random effects proportional odds model. Biometrics 58 316–323.
  • Lehmann, E.L. (1975). Nonparametrics: Statistical Methods Based on Ranks. Holden-Day, San Francisco.
  • Mantel, N. and Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of National Cancer Institution 22 719–748.
  • Nelder, J. A., Lee, Y. and Pawitan, Y. (2006). Generalized Linear Models with Random Effects: A Unified Approach via H-Likelihood. Chapman & Hall, London.
  • Olkin, I. and Liu, R. (2003). A bivariate Beta distribution. Statist. Probab. Lett. 62 407–412.
  • Sidik, K. and Jonkman, J. N. (2002). A simple confidence interval for meta-analysis. Statistics in Medicine 21 3153–3159.
  • Sidik, K. and Jonkman, J. N. (2007). A comparison of heterogeneity variance estimators in combining results of studies. Statistics in Medicine 26 1964–1981.
  • Sutton, A. J. and Higgins, J. P. T. (2008). Recent developments in meta-analysis. Statistics in Medicine 27 625–650.
  • Tian, L., Cai, T., Pfeffer, M. A., Piankov, N., Cremieux, P. and Wei, L. J. (2009). Exact and efficient inference procedure for meta-analysis and its application to the analysis of independent 2×2 tables with all available data but without artificial continuity correction. Biostatistics 10 275–281.
  • Viechtbauer, W. (2007). Confidence intervals for the amount of heterogeneity in meta-analysis. Statistics in Medicine 26 37–52.
  • Wang, R., Tian, L., Cai, T. and Wei, L. J. (2009). Supplement to “Nonparametric inference procedure for percentiles of the random effects distribution in meta analysis.” DOI: 10.1214/09-AOAS280SUPP.
  • Wu, C. F. J. (1986). Jackknife, bootstrap, and other resampling methods in regression analysis. Ann. Statist. 14 1261–1295.
  • Yusuf, S., Peto, R., Lewis, J., Collins, R. and Sleight, P. (1985). Beta blockade during and after myocardial infarction: An overview of the randomised trials. Progress in Cardiovascular Diseases 27 335–371.
  • Zeng, D. and Lin, D. Y. (2007). Maximum likelihood estimation in semiparametric regression models with censored data. J. R. Statist. Soc. Ser. B Stat. Methodol. 69 507–564.
  • Zeng, D., Lin, D. Y. and Lin, X. (2008). Semiparametric transformation models with random effects for clustered failure time data. Statist. Sinica 18 355–377.

Supplemental materials