The Annals of Statistics

Sharp bounds on the variance in randomized experiments

Peter M. Aronow, Donald P. Green, and Donald K. K. Lee

Full-text: Open access


We propose a consistent estimator of sharp bounds on the variance of the difference-in-means estimator in completely randomized experiments. Generalizing Robins [Stat. Med. 7 (1988) 773–785], our results resolve a well-known identification problem in causal inference posed by Neyman [Statist. Sci. 5 (1990) 465–472. Reprint of the original 1923 paper]. A practical implication of our results is that the upper bound estimator facilitates the asymptotically narrowest conservative Wald-type confidence intervals, with applications in randomized controlled and clinical trials.

Article information

Ann. Statist., Volume 42, Number 3 (2014), 850-871.

First available in Project Euclid: 20 May 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62A01: Foundations and philosophical topics
Secondary: 62D99: None of the above, but in this section 62G15: Tolerance and confidence regions

Causal inference finite populations potential outcomes randomized experiments variance estimation


Aronow, Peter M.; Green, Donald P.; Lee, Donald K. K. Sharp bounds on the variance in randomized experiments. Ann. Statist. 42 (2014), no. 3, 850--871. doi:10.1214/13-AOS1200.

Export citation


  • [1] Cochran, W. G. (1977). Sampling Techniques, 3rd ed. Wiley, New York.
  • [2] Copas, J. B. (1973). Randomization models for the matched and unmatched $2\times 2$ tables. Biometrika 60 467–476.
  • [3] Freedman, D. A. (2008). On regression adjustments in experiments with several treatments. Ann. Appl. Stat. 2 176–196.
  • [4] Freedman, D. A., Pisani, R. and Purves, R. A. (2007). Statistics, 4th ed. Norton, New York.
  • [5] Gadbury, G. L. (2001). Randomization inference and bias of standard errors. Amer. Statist. 55 310–313.
  • [6] Gadbury, G. L., Iyer, H. K. and Albert, J. M. (2004). Individual treatment effects in randomized trials with binary outcomes. J. Statist. Plann. Inference 121 163–174.
  • [7] Gerber, A. S. and Green, D. P. (2012). Field Experiments: Design, Analysis, and Interpretation. Norton, New York.
  • [8] Harrison, B. F. and Michelson, M. R. (2012). Not that there’s anything wrong with that: The effect of personalized appeals on marriage equality campaigns. Political Behavior 34 325–344.
  • [9] Heckman, J. J., Smith, J. and Clements, N. (1997). Making the most out of programme evaluations and social experiments: Accounting for heterogeneity in programme impacts. Rev. Econom. Stud. 64 487–535.
  • [10] Isaki, C. T. and Fuller, W. A. (1982). Survey design under the regression superpopulation model. J. Amer. Statist. Assoc. 77 89–96.
  • [11] Lehmann, E. L. (1966). Some concepts of dependence. Ann. Math. Statist. 37 1137–1153.
  • [12] Lin, W. (2013). Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique. Ann. Appl. Stat. 7 295–318.
  • [13] Neyman, J. S. (1990). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statist. Sci. 5 465–472. Reprint of the original 1923 paper.
  • [14] Robins, J. M. (1988). Confidence intervals for causal parameters. Stat. Med. 7 773–785.
  • [15] Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Ann. Statist. 6 34–58.
  • [16] Samii, C. and Aronow, P. M. (2012). On equivalencies between design-based and regression-based variance estimators for randomized experiments. Statist. Probab. Lett. 82 365–370.
  • [17] Tchen, A. H. (1980). Inequalities for distributions with given marginals. Ann. Probab. 8 814–827.
  • [18] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
  • [19] Zhang, Z., Wang, C., Nie, L. and Soon, G. (2013). Assessing the heterogeneity of treatment effects via potential outcomes of individual patients. J. Roy. Statist. Soc. Ser. C 62 687–704.