Statistical Science

Block-Conditional Missing at Random Models for Missing Data

Yan Zhou, Roderick J. A. Little, and John D. Kalbfleisch

Full-text: Open access

Abstract

Two major ideas in the analysis of missing data are (a) the EM algorithm [Dempster, Laird and Rubin, J. Roy. Statist. Soc. Ser. B 39 (1977) 1–38] for maximum likelihood (ML) estimation, and (b) the formulation of models for the joint distribution of the data Z and missing data indicators M, and associated “missing at random” (MAR) condition under which a model for M is unnecessary [Rubin, Biometrika 63 (1976) 581–592]. Most previous work has treated Z and M as single blocks, yielding selection or pattern-mixture models depending on how their joint distribution is factorized. This paper explores “block-sequential” models that interleave subsets of the variables and their missing data indicators, and then make parameter restrictions based on assumptions in each block. These include models that are not MAR. We examine a subclass of block-sequential models we call block-conditional MAR (BCMAR) models, and an associated block-monotone reduced likelihood strategy that typically yields consistent estimates by selectively discarding some data. Alternatively, full ML estimation can often be achieved via the EM algorithm. We examine in some detail BCMAR models for the case of two multinomially distributed categorical variables, and a two block structure where the first block is categorical and the second block arises from a (possibly multivariate) exponential family distribution.

Article information

Source
Statist. Sci., Volume 25, Number 4 (2010), 517-532.

Dates
First available in Project Euclid: 14 March 2011

Permanent link to this document
https://projecteuclid.org/euclid.ss/1300108235

Digital Object Identifier
doi:10.1214/10-STS344

Mathematical Reviews number (MathSciNet)
MR2807768

Zentralblatt MATH identifier
1329.62039

Keywords
Block-sequential missing data models block-conditional MAR models EM algorithm categorical data

Citation

Zhou, Yan; Little, Roderick J. A.; Kalbfleisch, John D. Block-Conditional Missing at Random Models for Missing Data. Statist. Sci. 25 (2010), no. 4, 517--532. doi:10.1214/10-STS344. https://projecteuclid.org/euclid.ss/1300108235


Export citation

References

  • Baker, S. G. (1995). Marginal regression for repeated binary data with outcome subject to non-ignorable nonresponse. Biometrics 51 1042–1052.
  • Baker, S. and Laird, N. (1985). Categorical response subject to nonresponse. Technical Report, Dept. Biostatistics, Harvard School of Public Health, Boston, MA.
  • Birmingham, J. and Fitzmaurice, G. M. (2002). A pattern-mixture model for longitudinal binary responses with nonignorable nonresponse. Biometrics 58 989–996.
  • Bishop, Y. M. M., Fienberg, S. E. and Holland, P. W. (1975). Discrete Multivariate Analysis: Theory and Practice. MIT Press, Cambridge, MA.
  • Chen, T. and Fienberg, S. E. (1974). Two-dimensional contingency tables with both completely and partially classified data. Biometrics 30 629–642.
  • Cox, D. R. (1975). Partial likelihood. Biometrika 62 269–276.
  • Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Roy. Statist. Soc. Ser. B. 39 1–38.
  • Ekholm, A. and Skinner, C. (1998). The muscatine children’s obesity data reanalysed using pattern mixture models. J. Roy. Statist. Soc. Ser. C. 47 251–263.
  • Fay, R. E. (1986). Causal models for patterns of nonresponse. J. Amer. Statist. Assoc. 81 354–365.
  • Frangakis, C. E. and Rubin, D. B. (1999). Addressing complications of intention-to-treat analysis in the combined presence of all-or-none treatment-noncompliance and subsequent missing outcomes. Biometrika 86 365–379.
  • Fuchs, C. (1982). Maximum likelihood estimation and model selection in contingency tables with missing data. J. Amer. Statist. Assoc. 77 270–278.
  • Hartley, H. O. (1958). Maximum likelihood estimation from incomplete data. Biometrics 14 174–194.
  • Heckman, J. I. (1976). The common structure of statistical models of truncation, sample selection and limited dependent variables, and a simple estimator for such models. Ann. Econ. Soc. Meas. 5 475–492.
  • Lipsitz, S. R., Ibrahim, J. G. and Zhao, L. P. (1999). A weighted estimating equation for missing covariate data with properites similar to maximum likelihood. J. Amer. Statist. Assoc. 94 1147–1160.
  • Lipsitz, S. R., Parzen, M. and Molenberghs, G. (1998). Obtaining the maximum likelihood estimates in incomplete R×C contingency tables using a Poisson generalized linear model. J. Comput. Graph. Statist. 7 356–376.
  • Little, R. J. A. (1993). Pattern-mixture models for multivariate incomplete data. J. Amer. Statist. Assoc. 88 125–134.
  • Little, R. J. A. (1995). Modeling the drop-out mechanism in repeated-measures studies. J. Amer. Statist. Assoc. 90 1112–1121.
  • Little, R. J. A. and Rubin, D. B. (2002). Statistical Analysis with Missing Data. Wiley, Hoboken, NJ.
  • Little, R. J. A. and Zhang, N. (2011). Subsample ignorable likelihood for regression analysis with missing data. J. Roy. Statist. Soc. Ser. C 60. To appear.
  • Meng, X.-L. and Rubin, D. B. (1993). Maximum likelihood estimation via the ECM algorithm: A general framework. Biometrika 80 267–278.
  • Peng, Y. H., Little, R. J. A. and Raghunathan, T. E. (2004). An extended general location model for causal inferences from data subject to noncompliance and missing values. Biometrics 60 598–607.
  • Robins, J. M. (1997). Non-response models for the analysis of non-monotone non-ignorable missing data. Stat. Med. 16 21–37.
  • Robins, J. M. and Gill, R. (1997). Non-response models for the analysis of non-monotone ignorable missing data. Stat. Med. 16 39–56.
  • Rubin, D. B. (1976). Inference and missing data (with discussion). Biometrika 63 581–592.
  • Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. Wiley, New York.
  • Rubin, D. B., Stern, H. and Vehovar, V. (1995). Handling “don’t know” survey responses: The case of the Slovenian plebiscite. J. Amer. Statist. Assoc. 90 822–828.
  • Woolson, R. F. and Clarke, W. R. (1984). Analysis of categorical incomplete longitudinal data. J. Roy. Statist. Soc. Ser. A 147 87–99.