The Annals of Applied Statistics

Flexible covariate-adjusted exact tests of randomized treatment effects with application to a trial of HIV education

Alisa J. Stephens, Eric J. Tchetgen Tchetgen, and Victor De Gruttola

Full-text: Open access

Abstract

The primary goal of randomized trials is to compare the effects of different interventions on some outcome of interest. In addition to the treatment assignment and outcome, data on baseline covariates, such as demographic characteristics or biomarker measurements, are typically collected. Incorporating such auxiliary covariates in the analysis of randomized trials can increase power, but questions remain about how to preserve type I error when incorporating such covariates in a flexible way, particularly when the number of randomized units is small. Using the Young Citizens study, a cluster-randomized trial of an educational intervention to promote HIV awareness, we compare several methods to evaluate intervention effects when baseline covariates are incorporated adaptively. To ascertain the validity of the methods shown in small samples, extensive simulation studies were conducted. We demonstrate that randomization inference preserves type I error under model selection while tests based on asymptotic theory may yield invalid results. We also demonstrate that covariate adjustment generally increases power, except at extremely small sample sizes using liberal selection procedures. Although shown within the context of HIV prevention research, our conclusions have important implications for maximizing efficiency and robustness in randomized trials with small samples across disciplines.

Article information

Source
Ann. Appl. Stat., Volume 7, Number 4 (2013), 2106-2137.

Dates
First available in Project Euclid: 23 December 2013

Permanent link to this document
https://projecteuclid.org/euclid.aoas/1387823312

Digital Object Identifier
doi:10.1214/13-AOAS679

Mathematical Reviews number (MathSciNet)
MR3161715

Zentralblatt MATH identifier
1283.62238

Keywords
Randomized trials exact tests covariate adjustment model selection

Citation

Stephens, Alisa J.; Tchetgen Tchetgen, Eric J.; De Gruttola, Victor. Flexible covariate-adjusted exact tests of randomized treatment effects with application to a trial of HIV education. Ann. Appl. Stat. 7 (2013), no. 4, 2106--2137. doi:10.1214/13-AOAS679. https://projecteuclid.org/euclid.aoas/1387823312


Export citation

References

  • Bickel, P. J. and van Zwet, W. R. (1978). Asymptotic expansions for the power of distribution free tests in the two-sample problem. Ann. Statist. 6 937–1004.
  • Black, P. (2005). Greedy algorithm. In Dictionary of Algorithms and Data Structures [online] (V. Pieterse and P. E. Black, eds.), 2 February 2005 (accessed May 2012).
  • Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression. Ann. Statist. 32 407–499.
  • Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360.
  • Fisher, R. A. (1935). The Design of Experiments. Oliver and Boyd, Edinburgh.
  • Gail, M. H., Tan, W. Y. and Piantadosi, S. (1988). Tests for no treatment effect in randomized clinical trials. Biometrika 75 57–64.
  • Klar, N. and Donner, A. (2000). Design and Analysis of Cluster Randomization Trials in Health Research. Hodder Arnold, London.
  • Pauler, D. K. (1998). The Schwarz criterion and related methods for normal linear models. Biometrika 85 13–27.
  • Rosenbaum, P. R. (2002). Covariance adjustment in randomized experiments and observational studies. Statist. Sci. 17 286–327.
  • Sofer, T., Dicker, L. and Lin, X. (2012). Variable selection for high-dimensional multivariate outcomes. Unpublished manuscript.
  • Stephens, A. J., Tchetgen Tchetgen, E. J. andDe Gruttola, V. (2012). Augmented GEE for improving efficiency of inferences in cluster randomized trials by leveraging cluster and individual-level covariates. Stat. Med. 31 915–930.
  • Stephens, A. J., Tchetgen Tchetgen, E. J. andDe Gruttola, V. (2013a). Locally efficient estimation of marginal treatment effects using auxiliary covariates in randomized trials with correlated outcomes. Unpublished manuscript.
  • Stephens, A. J., Tchetgen Tchetgen, E. J. andDe Gruttola, V. (2013b). Supplement to “Flexible covariate-adjusted exact tests of randomized treatment effects with application to a trial of HIV education.” DOI:10.1214/13-AOAS679SUPP.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58 267–288.
  • Tsiatis, A. A., Davidian, M., Zhang, M. and Lu, X. (2008). Covariate adjustment for two-sample treatment comparisons in randomized clinical trials: A principled yet flexible approach. Stat. Med. 27 4658–4677.
  • van der Laan, M. J. and Robins, J. M. (2003). Unified Methods for Censored Longitudinal Data and Causality. Springer, New York.
  • Wang, Y.-G. and Carey, V. (2003). Working correlation structure misspecification, estimation and covariate design: Implications for generalised estimating equations performance. Biometrika 90 29–41.
  • Zeger, S. L. and Liang, K. Y. (1986). Longitudinal data analysis for discrete and continuous outcomes. Biometrics 42 121–130.
  • Zhang, C. H. (2010). Nearly unbiased variable selection under minimax concave penalty. J. Amer. Statist. Assoc. 101 1418–1429.
  • Zhang, M., Tsiatis, A. A. and Davidian, M. (2008). Improving efficiency of inferences in randomized clinical trials using auxiliary covariates. Biometrics 64 707–715.
  • Zou, H. (2006). The adaptive Lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418–1429.

Supplemental materials

  • Supplementary material: Supplement to “Flexible covariate-adjusted exact tests of randomized treatment effects with application to a trial of HIV education”. Supplement A: Small sample adjustment of Bickel van Zwet (1978). Function definitions in Bickel and van Zwet (1978) small-sample approximation. Supplement B: Simulation study tables—independent outcomes. Type I error and power of covariate-adjusted tests in independent outcomes. Supplement C: Simulation study tables—dependent outcomes. Type I Error under low correlation and power under low correlation and high correlation of covariate-adjusted tests for dependent outcomes.