Bayesian Analysis

Control of Type I Error Rates in Bayesian Sequential Designs

Haolun Shi and Guosheng Yin

Full-text: Open access


Bayesian approaches to phase II clinical trial designs are usually based on the posterior distribution of the parameter of interest and calibration of certain threshold for decision making. If the posterior probability is computed and assessed in a sequential manner, the design may involve the problem of multiplicity, which, however, is often a neglected aspect in Bayesian trial designs. To effectively maintain the overall type I error rate, we propose solutions to the problem of multiplicity for Bayesian sequential designs and, in particular, the determination of the cutoff boundaries for the posterior probabilities. We present both theoretical and numerical methods for finding the optimal posterior probability boundaries with α-spending functions that mimic those of the frequentist group sequential designs. The theoretical approach is based on the asymptotic properties of the posterior probability, which establishes a connection between the Bayesian trial design and the frequentist group sequential method. The numerical approach uses a sandwich-type searching algorithm, which immensely reduces the computational burden. We apply least-square fitting to find the α-spending function closest to the target. We discuss the application of our method to single-arm and double-arm cases with binary and normal endpoints, respectively, and provide a real trial example for each case.

Article information

Bayesian Anal., Volume 14, Number 2 (2019), 399-425.

First available in Project Euclid: 23 June 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62C10: Bayesian problems; characterization of Bayes procedures
Secondary: 62P10: Applications to biology and medical sciences

Bayesian design group sequential method multiple testing phase II clinical trial posterior probability type I error rate

Creative Commons Attribution 4.0 International License.


Shi, Haolun; Yin, Guosheng. Control of Type I Error Rates in Bayesian Sequential Designs. Bayesian Anal. 14 (2019), no. 2, 399--425. doi:10.1214/18-BA1109.

Export citation


  • Barber, S. and Jennison, C. (2002). “Optimal asymmetric one-sided group sequential tests.” Biometrika, 89(1): 49–60.
  • Berger, J. O. and Berry, D. A. (1988). “Statistical analysis and the illusion of objectivity.” American Scientist, 76(1): 159–165.
  • Berry, D. A. (2006). “Bayesian clinical trials.” Nature Reviews Drug Discovery, 5(1): 218–226.
  • Berry, D. A. (2011). “Adaptive clinical trials in oncology.” Nature Reviews Drug Discovery, 9(1): 199–207.
  • Berry, D. A. and Hochberg, Y. (1999). “Bayesian perspectives on multiple comparisons.” Journal of Statistical Planning and Inference, 82(1): 215–227.
  • Carlin, B. P., Kadane, J. B., and Gelfand, A. E. (1998). “Approaches for optimal sequential decision analysis in clinical trials.” Biometrics, 54(3): 964–975.
  • Christen, J. A., Müller, P., Wathen, J. K., and Wolf, J. (2004). “Bayesian randomized clinical trials: A decision-theoretic sequential design.” Canadian Journal of Statistics, 32(4): 387–402.
  • Dudley, R. M. and Haughton, D. (2002). “Asymptotic normality with small relative errors of posterior probabilities of half-spaces.” The Annals of Statistics, 30(5): 1311–1344.
  • Eales, J. D. and Jennison, C. (1992). “An improved method for deriving optimal one-sided group sequential tests.” Biometrika, 79(1): 13–24.
  • Efron, B. (1986). “Why isn’t everyone a Bayesian.” The American Statistician, 40(1): 1–5.
  • Efron, B. (2005). “Bayesians, frequentists, and scientists.” Journal of the American Statistical Association, 100(1): 1–5.
  • Gopalan, R. and Berry, D. A. (1998). “Bayesian multiple comparisons using Dirichlet process priors.” Journal of the American Statistical Association, 93(443): 1130–1139.
  • Gsponer, T., Gerber, F., Bornkamp, B., Ohlssen, D., Vandemeulebroecke, M., and Schmidli, H. (2014). “A practical guide to Bayesian group sequential designs.” Pharmaceutical Statisics, 13(1): 71–80.
  • Guindani, M., Müller, P., and Zhang, S. (2009). “A Bayesian discovery procedure.” Journal of the Royal Statistical Society. Series B, 71(5): 905–925.
  • Guo, M. and Heitjan, D. F. (2010). “Multiplicity-calibrated Bayesian hypothesis tests.” Biostatistics, 11(3): 473–483.
  • Heitjan, D. F. (1997). “Bayesian interim analysis of phase II cancer clinical trials.” Statistics in Medicine, 16(1): 1791–1802.
  • Hobbs, B. P., Carlin, B. P., Mandrekar, S. J., and Sargent, D. J. (2011). “Hierarchical commensurate and power prior models for adaptive incorporation of historical information in clinical trials.” Biometrics, 67(1): 1047–1056.
  • Jennison, C. and Turnbull, B. W. (1989). “Interim analyses: the repeated confidence interval approach.” Journal of the Royal Statistical Society. Series B, 51(1): 305–361.
  • Jennison, C. and Turnbull, B. W. (1997). “Group sequential analysis incorporating covariate information.” Journal of the American Statistical Association, 92(440): 1330–1341.
  • Jennison, C. and Turnbull, B. W. (2000). Group Sequential Methods with Applications to Clinical Trials. Boca Raton: Chapman & Hall/CRC.
  • Labbe, A. and Thompson, M. E. (2007). “Multiple testing using the posterior probabilities of directional alternatives, with application to genomic studies.” The Canadian Journal of Statistics, 35(1): 51–68.
  • Lee, J. J. and Liu, D. D. (2008). “A predictive probability design for phase II cancer clinical trials.” Clinical Trials, 5(2): 93–106.
  • Lewis, R. J. and Berry, D. A. (1994). “Group sequential clinical trials, A classical evaluation of Bayesian decision-theoretic designs.” Journal of the American Statistical Association, 89(1): 1528–1534.
  • Maki, R. G., Wathen, J. K., Patel, S. R., Priebat, D. A., Okuno, S. H., Samuels, B., Fanucchi, M., Harmon, D. C., Schuetze, S. M., Reinke, D., Thall, P. F., Benjamin, R. S., Baker, L. H., and Hensley, M. L. (2007). “Randomized phase II study of gemcitabine and docetaxel compared with gemcitabine alone in patients with metastatic soft tissue sarcomas: results of sarcoma alliance for research through collaboration study 002.” Journal of Clinical Oncology, 25(1): 2755–2763.
  • Morita, S., Müller, P., and Thall, P. F. (2008). “Determining the effective sample size of a parametric prior.” Biometrics, 64(2): 595–602.
  • Müller, P., Berry, D. A., Grieve, A. P., Smith, M., and Krams, M. (2007). “Simulation-based sequential Bayesian design.” Journal of Statistical Planning and Inference, 137(1): 3140–3150.
  • Murray, T. A., Thall, P. F., and Yuan, Y. (2016). “Utility-based designs for randomized comparative trials with categorical outcomes.” Statistics in Medicine, 35(24): 4285–4305.
  • Murray, T. A., Thall, P. F., Yuan, Y., McAvoy, S., and Gomez, D. R. (2017). “Robust treatment comparison based on utilities of semi-competing risks in non-small-cell lung cancer.” Journal of the American Statistical Association, 112(517): 11–23.
  • O’Brien, P. C. and Fleming, T. R. (1979). “A multiple testing procedure for clinical trials.” Biometrics, 35(3): 549–556.
  • Pocock, S. J. (1977). “Group sequential methods in the design and analysis of clinical trials.” Biometrika, 64(2): 191–199.
  • Rosner, G. L. and Berry, D. A. (1995). “A Bayesian group sequential design for a multiple arm randomized clinical trial.” Statistics in Medicine, 14(14): 381–394.
  • Scharfstein, D. O., Tsiatis, A. A., and Robins, J. M. (1997). “Semiparametric efficiency and its implication on the design and analysis of group sequential studies.” 92(440): 1342–1350.
  • Scott, J. G. and Berger, J. O. (2006). “An exploration of aspects of Bayesian multiple testing.” Journal of Statistical Planning and Inference, 136(7): 2144–2162.
  • Shi, H. and Yin, G. (2018). “Supplementary Material of the Control of type I error rates in Bayesian sequential designs” Bayesian Analysis.
  • Thall, P. F. and Simon, R. (1994). “Practical Bayesian guidelines for phase IIB clinical trials.” Biometrics, 50(2): 337–349.
  • Thall, P. F., Simon, R., and Estey, E. H. (1995). “Bayesian sequential monitoring designs for single-arm clinical trials with multiple outcomes.” Statistics in Medicine, 14(1): 357–379.
  • Ventz, S. and Trippa, L. (2015). “Bayesian designs and the control of frequentist characteristics: A practical solution.” Biometrics, 71(1): 218–226.
  • Wang, S. K. and Tsiatis, A. A. (1987). “Approximately optimal one-parameter boundaries for group sequential trials.” Biometrics, 43(1): 193–199.
  • Wason, J. M. S., Abraham, S. E., Baird, R. D., Gournaris, I., Vallier, A., Brenton, J. D., Earl, H. M., and Mander, A. P. (2015). “A Bayesian adaptive design for biomarker trials with linked treatments.” British Journal of Cancer, 113(5): 699–705.
  • Wathen, J. K. and Thall, P. F. (2008). “Bayesian adaptive model selection for optimizing group sequential clinical trials.” Statistics in Medicine, 27(27): 5586–5604.
  • Yin, G. (2012). Clinical Trial Design: Bayesian and Frequentist Adaptive Methods. Hoboken: John Willey & Sons, Inc.
  • Zhu, H. and Yu, Q. (2017). “A Bayesian sequential design using alpha spending function to control type I error.” Statistical Methods in Medical Research, 26(5): 2184–2196.

Supplemental materials