Annals of Applied Statistics

Design of vaccine trials during outbreaks with and without a delayed vaccination comparator

Natalie E. Dean, M. Elizabeth Halloran, and Ira M. Longini

Full-text: Open access


Conducting vaccine efficacy trials during outbreaks of emerging pathogens poses particular challenges. The “Ebola ça suffit” trial in Guinea used a novel ring vaccination cluster randomized design to target populations at highest risk of infection. Another key feature of the trial was the use of a delayed vaccination arm as a comparator, in which clusters were randomized to immediate vaccination or vaccination 21 days later. This approach, chosen to improve ethical acceptability of the trial, complicates the statistical analysis as participants in the comparison arm are eventually protected by vaccine. Furthermore, for infectious diseases, we observe time of illness onset and not time of infection, and we may not know the time required for the vaccinee to develop a protective immune response. As a result, including events observed shortly after vaccination may bias the per protocol estimate of vaccine efficacy. We provide a framework for approximating the bias and power of any given analysis period as functions of the background infection hazard rate, disease incubation period, and vaccine immune response. We use this framework to provide recommendations for designing standard vaccine efficacy trials and trials with a delayed vaccination comparator. Briefly, narrower analysis periods within the correct window can minimize or eliminate bias but may suffer from reduced power. Designs should be reasonably robust to misspecification of the incubation period and time to develop a vaccine immune response.

Article information

Ann. Appl. Stat., Volume 12, Number 1 (2018), 330-347.

Received: September 2016
Revised: August 2017
First available in Project Euclid: 9 March 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Vaccine trial infectious diseases per protocol


Dean, Natalie E.; Halloran, M. Elizabeth; Longini, Ira M. Design of vaccine trials during outbreaks with and without a delayed vaccination comparator. Ann. Appl. Stat. 12 (2018), no. 1, 330--347. doi:10.1214/17-AOAS1095.

Export citation


  • Bellan, S. E., Pulliam, J. R. C., Pearson, C. A. B., Champredon, D., Fox, S. J., Skrip, L., Galvani, A. P., Gambhir, M., Lopman, B. A., Porco, T. C., Meyers, L. A. and Dushoff, J. (2015). Statistical power and validity of Ebola vaccine trials in Sierra Leone: A simulation study of trial design and analysis. Lancet, Infect. Dis. 15 703–710.
  • Ebola ça Suffit Ring Vaccination Trial Consortium (2015). The ring vaccination trial: A novel cluster randomised controlled trial design to evaluate vaccine efficacy and effectiveness during outbreaks, with special reference to Ebola. Br. Med. J. 351 h3740.
  • Camacho, A., Eggo, R. M., Funk, S., Watson, C. H., Kucharski, A. J. and Edmunds, W. J. (2015). Estimating the probability of demonstrating vaccine efficacy in the declining Ebola epidemic: A Bayesian modelling approach. BMJ Open 5 e009346.
  • Dean, N., Halloran, M. E. and Longini, I. M. (2018). Supplement to “Design of vaccine trials during outbreaks with and without a delayed vaccination comparator.” DOI:10.1214/17-AOAS1095SUPP.
  • Gilbert, P. B., Berger, J. O., Stablein, D., Becker, S., Essex, M., Hammer, S. M., Kim, J. H. and DeGruttola, V. G. (2011). Statistical interpretation of the RV144 HIV vaccine efficacy trial in Thailand: A case study for statistical issues in efficacy trials. J. Infect. Dis. 203 969–975.
  • Halloran, M. E., Longini, I. M. Jr. and Struchiner, C. J. (2010). Design and Analysis of Vaccine Studies. Springer, New York.
  • Halloran, M. E., Haber, M., Longini, I. M. and Struchiner, C. J. (1991). Direct and indirect effects in vaccine efficacy and effectiveness. Am. J. Epidemiol. 133 323–331.
  • Henao-Restrepo, A. M., Longini, I. M., Egger, M., Dean, N. E., Edmunds, W. J., Camacho, A., Carroll, M. W., Doumbia, M., Draguez, B., Duraffour, S., Enwere, G., Grais, R., Gunther, S., Hossmann, S., Konde, M. K., Kone, S., Kuisma, E., Levine, M. M., Mandal, S., Norheim, G., Riveros, X., Soumah, A., Trelle, S., Vicari, A. S., Watson, C. H., Keita, S., Kieny, M.-P. P. and Røttingen, J. A. (2015). Efficacy and effectiveness of an rVSV-vectored vaccine expressing Ebola surface glycoprotein: Interim results from the Guinea ring vaccination cluster-randomised trial. Lancet 386 857–866.
  • Henao-Restrepo, A. M., Camacho, A., Longini, I. M., Watson, C. H., Edmunds, W. J., Egger, M., Carroll, M. W., Dean, N. E., Datta, I., Doumbia, M., Draguez, B., Duraffour, S., Enwere, G., Grais, R., Gunther, S., Gsell, P.-S., Hossmann, S., Watle, S. V., Konde, M. K., Keita, S., Kone, S., Kuisma, E., Levine, M. M., Mandal, S., Mauget, T., Norheim, G., Riveros, X., Soumah, A., Trelle, S., Vicari, A. S., Røttingen, J. A. and Kieny, M.-P. (2017). Efficacy and effectiveness of an rVSV-vectored vaccine preventing Ebola virus disease: Final results from the Guinea ring vaccination, open-label, cluster-randomised trial (Ebola ça suffit!). Lancet 389 505–518.
  • Hernán, M. A. (2013). The hazards of hazard ratios. Epidemiology 21 13–15.
  • Horne, A. D., Lachenbruch, P. A. and Goldenthal, K. L. (2001). Intent-to-treat analysis and preventive vaccine efficacy. Vaccine 19 319–326.
  • Hussey, M. A. and Hughes, J. P. (2007). Design and analysis of stepped wedge cluster randomized trials. Contemp. Clin. Trials 28 182–191.
  • Kieny, M. P. and Salama, P. (2017). WHO R&D blueprint: A global coordination mechanism for R&D preparedness. Lancet 389 2469–2470.
  • RTSS Clinical Trials Partnership (2015). Efficacy and safety of RTS, S/AS01 malaria vaccine with or without a booster dose in infants and children in Africa: Final results of a phase 3, individually randomised, controlled trial. Lancet 386 31–45.
  • Ridout, M. S., Demétrio, C. G. and Firth, D. (1999). Estimating intraclass correlation for binary data. Biometrics 55 137–148.
  • Rosner, B. (2010). Fundamentals of Biostatistics, 7th ed. Cengage Learning, Boston.
  • R Core Team (2015). A Language and Environment for Statistical Computing. Available at
  • Therneau, T. M. (2015). Package ‘survival’: Survival analysis. R package. Available at
  • Xu, R. and O’Quigley, J. (2000). Estimating average regression effect under non-proportional hazards. Biostatistics 1 423–439.
  • Zucker, D. M. and Lakatos, E. (1990). Weighted log rank type statistics for comparing survival curves when there is a time lag in the effectiveness of treatment. Biometrika 77 853–864.

Supplemental materials

  • Supplement to “Design of vaccine trials during outbreaks with and without a delayed vaccination comparator”. The supplementary materials contains additional plots and tables under a range of scenarios. Supporting R code is also provided to produce estimates and plots.