Statistical Science

Oracle, Multiple Robust and Multipurpose Calibration in a Missing Response Problem

Kwun Chuen Gary Chan and Sheung Chi Phillip Yam

Full-text: Open access


In the presence of a missing response, reweighting the complete case subsample by the inverse of nonmissing probability is both intuitive and easy to implement. When the population totals of some auxiliary variables are known and when the inclusion probabilities are known by design, survey statisticians have developed calibration methods for improving efficiencies of the inverse probability weighting estimators and the methods can be applied to missing data analysis. Model-based calibration has been proposed in the survey sampling literature, where multidimensional auxiliary variables are first summarized into a predictor function from a working regression model. Usually, one working model is being proposed for each parameter of interest and results in different sets of calibration weights for estimating different parameters. This paper considers calibration using multiple working regression models for estimating a single or multiple parameters. Contrary to a common belief that overfitting hurts efficiency, we present three rather unexpected results. First, when the missing probability is correctly specified and multiple working regression models for the conditional mean are posited, calibration enjoys an oracle property: the same semiparametric efficiency bound is attained as if the true outcome model is known in advance. Second, when the missing data mechanism is misspecified, calibration can still be a consistent estimator when any one of the outcome regression models is correctly specified. Third, a common set of calibration weights can be used to improve efficiency in estimating multiple parameters of interest and can simultaneously attain semiparametric efficiency bounds for all parameters of interest. We provide connections of a wide class of calibration estimators, constructed based on generalized empirical likelihood, to many existing estimators in biostatistics, econometrics and survey sampling and perform simulation studies to show that the finite sample properties of calibration estimators conform well with the theoretical results being studied.

Article information

Statist. Sci., Volume 29, Number 3 (2014), 380-396.

First available in Project Euclid: 23 September 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Generalized empirical likelihood model misspecification missing data robustness


Chan, Kwun Chuen Gary; Yam, Sheung Chi Phillip. Oracle, Multiple Robust and Multipurpose Calibration in a Missing Response Problem. Statist. Sci. 29 (2014), no. 3, 380--396. doi:10.1214/13-STS461.

Export citation


  • Bang, H. and Robins, J. M. (2005). Doubly robust estimation in missing data and causal inference models. Biometrics 61 962–972.
  • Breslow, N. E., Lumley, T., Ballantyne, C. M., Chambless, L. E. and Kulich, M. (2009). Improved Horvitz–Thompson estimation of model parameters from two-phase stratified samples: Applications in epidemiology. Statistics in Biosciences 1 32–49.
  • Cassel, C. M., Särndal, C. E. and Wretman, J. H. (1976). Some results on generalized difference estimation and generalized regression estimation for finite populations. Biometrika 63 615–620.
  • Chan, K. C. G. (2012). Uniform improvement of empirical likelihood for missing response problem. Electron. J. Stat. 6 289–302.
  • Chan, K. C. G. (2013). A simple multiply robust estimator for missing response problem. Stat 2 143–149.
  • Chan, K. C. G. and Yam, S. C. P. (2014). Supplement to “Oracle, Multiple Robust and Multipurpose Calibration in a Missing Response Problem.” DOI:10.1214/13-STS461SUPP.
  • Chaussé, P. (2010). Computing generalized method of moments and generalized empirical likelihood with R. Journal of Statistical Software 34 1–35.
  • Chen, J. and Sitter, R. R. (1999). A pseudo empirical likelihood approach to the effective use of auxiliary information in complex surveys. Statist. Sinica 9 385–406.
  • Chen, J., Sitter, R. R. and Wu, C. (2002). Using empirical likelihood methods to obtain range restricted weights in regression estimators for surveys. Biometrika 89 230–237.
  • Cressie, N. and Read, T. R. C. (1984). Multinomial goodness-of-fit tests. J. Roy. Statist. Soc. Ser. B 46 440–464.
  • Deming, W. E. and Stephan, F. F. (1940). On a least squares adjustment of a sampled frequency table when the expected marginal totals are known. Ann. Math. Statist. 11 427–444.
  • Deville, J.-C. and Särndal, C.-E. (1992). Calibration estimators in survey sampling. J. Amer. Statist. Assoc. 87 376–382.
  • Deville, J. C., Särndal, C. E. and Sautory, O. (1993). Generalized raking procedures in survey sampling. J. Amer. Statist. Assoc. 88 1013–1020.
  • Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360.
  • Graham, B. S., De Xavier Pinto, C. C. and Egel, D. (2012). Inverse probability tilting for moment condition model with missing data. Rev. Econ. Stud. 79 1053–1079.
  • Hahn, J. (1998). On the role of the propensity score in efficient semiparametric estimation of average treatment effects. Econometrica 66 315–331.
  • Hainmueller, J. (2012). Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies. Political Analysis 20 25–46.
  • Han, P. and Wang, L. (2013). Estimation with missing data: Beyond double robustness. Biometrika 100 417–430.
  • Hansen, L. P. (1982). Large sample properties of generalized method of moments estimators. Econometrica 50 1029–1054.
  • Hansen, L. P., Heaton, J. and Yaron, A. (1996). Finite-sample properties of some alternative GMM estimators. J. Bus. Econom. Statist. 14 262–280.
  • Hellerstein, J. K. and Imbens, G. W. (1999). Imposing moment restrictions from auxiliary data by weighting. Rev. Econ. Statist. 81 1–14.
  • Horvitz, D. G. and Thompson, D. J. (1952). A generalization of sampling without replacement from a finite universe. J. Amer. Statist. Assoc. 47 663–685.
  • Imbens, G. W., Spady, R. H. and Johnson, P. (1998). Information-theoretic approaches to inference in moment condition models. Econometrica 66 333–357.
  • Kang, J. D. Y. and Schafer, J. L. (2007). Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Statist. Sci. 22 523–539.
  • Kim, J. K. (2009). Calibration estimation using empirical likelihood in survey sampling. Statist. Sinica 19 145–157.
  • Kitamura, Y. and Stutzer, M. (1997). An information-theoretic alternative to generalized method of moments estimation. Econometrica 65 861–874.
  • Kott, P. S. and Chang, T. (2010). Using calibration weighting to adjust for nonignorable unit nonresponse. J. Amer. Statist. Assoc. 105 1265–1275.
  • Lehmann, E. L. and Casella, G. (1998). Theory of Point Estimation, 2nd ed. Springer, New York.
  • Lindsay, B. G. and Qu, A. (2003). Inference functions and quadratic score tests. Statist. Sci. 18 394–410.
  • Lumley, T., Shaw, P. A. and Dai, J. Y. (2011). Connections between survey calibration estimators and semiparametric models for incomplete data. Internat. Statist. Rev. 79 200–220.
  • McCaffrey, D. F., Ridgeway, G. and Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods 9 403–425.
  • Newey, W. K. and McFadden, D. (1994). Large sample estimation and hypothesis testing. In Handbook of Econometrics, Vol. IV. Handbooks in Econom. 2 2111–2245. North-Holland, Amsterdam.
  • Newey, W. K. and Smith, R. J. (2004). Higher order properties of GMM and generalized empirical likelihood estimators. Econometrica 72 219–255.
  • Owen, A. B. (1988). Empirical likelihood ratio confidence intervals for a single functional. Biometrika 75 237–249.
  • Qin, J. and Lawless, J. (1994). Empirical likelihood and general estimating equations. Ann. Statist. 22 300–325.
  • Qin, J. and Zhang, B. (2007). Empirical-likelihood-based inference in missing response problems and its application in observational studies. J. R. Stat. Soc. Ser. B Stat. Methodol. 69 101–122.
  • Ridgeway, G. and McCaffrey, D. F. (2007). Comment: Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Statist. Sci. 22 540–543.
  • Robins, J. M. and Rotnitzky, A. (1995). Semiparametric efficiency in multivariate regression models with missing data. J. Amer. Statist. Assoc. 90 122–129.
  • Robins, J. M., Rotnitzky, A. and Zhao, L. P. (1994). Estimation of regression coefficients when some regressors are not always observed. J. Amer. Statist. Assoc. 89 846–866.
  • Saegusa, T. and Wellner, J. A. (2013). Weighted likelihood estimation under two-phase sampling. Ann. Statist. 41 269–295.
  • Scharfstein, D. O., Rotnitzky, A. and Robins, J. M. (1999). Adjusting for nonignorable drop-out using semiparametric nonresponse models. J. Amer. Statist. Assoc. 94 1096–1146.
  • Tan, Z. (2006). A distributional approach for causal inference using propensity scores. J. Amer. Statist. Assoc. 101 1619–1637.
  • Théberge, A. (1999). Extensions of calibration estimators in survey sampling. J. Amer. Statist. Assoc. 94 635–644.
  • White, H. (1982). Maximum likelihood estimation of misspecified models. Econometrica 50 1–25.
  • Wu, C. and Sitter, R. R. (2001). A model-calibration approach to using complete auxiliary information from survey data. J. Amer. Statist. Assoc. 96 185–193.
  • Zou, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418–1429.

Supplemental materials

  • Supplementary material: Proof of the Main Results. Online supplementary material is provided that includes a list of regularity conditions, the proofs of Lemma 1, Theorem 2 and Corollary 3, together with two technical lemmas that were needed to prove Lemma 1.