Statistical Science

Calibrated Bayes, for Statistics in General, and Missing Data in Particular

Roderick Little

Full-text: Open access


It is argued that the Calibrated Bayesian (CB) approach to statistical inference capitalizes on the strength of Bayesian and frequentist approaches to statistical inference. In the CB approach, inferences under a particular model are Bayesian, but frequentist methods are useful for model development and model checking. In this article the CB approach is outlined. Bayesian methods for missing data are then reviewed from a CB perspective. The basic theory of the Bayesian approach, and the closely related technique of multiple imputation, is described. Then applications of the Bayesian approach to normal models are described, both for monotone and nonmonotone missing data patterns. Sequential Regression Multivariate Imputation and Penalized Spline of Propensity Models are presented as two useful approaches for relaxing distributional assumptions.

Article information

Statist. Sci., Volume 26, Number 2 (2011), 162-174.

First available in Project Euclid: 1 August 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Maximum likelihood multiple imputation penalized splines propensity models sequential regression multivariate imputation


Little, Roderick. Calibrated Bayes, for Statistics in General, and Missing Data in Particular. Statist. Sci. 26 (2011), no. 2, 162--174. doi:10.1214/10-STS318.

Export citation


  • Abayomi, K., Gelman, A. and Levy, M. (2008). Diagnostics for multivariate imputations. Appl. Statist. 57 273–291.
  • Agresti, A. (2002). Categorical Data Analysis, 2nd ed. Wiley, New York.
  • Anderson, T. W. (1957). Maximum likelihood estimates for the multivariate normal distribution when some observations are missing. J. Amer. Statist. Assoc. 52 200–203.
  • Bang, H. and Robins, J. M. (2005). Doubly robust estimation in missing data and causal inference models. Biometrics 61 962–972.
  • Barnard, J. and Rubin, D. B. (1999). Small-sample degrees of freedom with multiple imputation. Biometrika 86 949–955.
  • Baum, L. E., Petrie, T., Soules, G. and Weiss, N. (1970). A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Statist. 41 164–171.
  • Box, G. E. P. (1980). Sampling and Bayes inference in scientific modelling and robustness (with discussion). J. Roy. Statist. Soc. Ser. A 143 383–430.
  • Daniels, M. J. and Hogan, J. W. (2008). Missing Data in Longitudinal Studies: Strategies for Bayesian Modeling and Sensitivity Analysis. CRC Press, New York.
  • Dawid, A. P. (1982). The well-calibrated Bayesian. J. Amer. Statist. Assoc. 77 605–610.
  • Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Roy. Statist. Soc. Ser. B 39 1–38.
  • Draper, D. (1995). Assessment and propagation of model uncertainty (with discussion). J. Roy. Statist. Soc. Ser. B 57 45–97.
  • Efron, B. (1986). Why isn’t everyone a Bayesian? (with discussion and rejoinder). Amer. Statist. 40 1–11.
  • Eilers, P. H. C. and Marx, B. D. (1996). Flexible smoothing with B-Splines and penalties. Statist. Sci. 11 89–121.
  • Gelman, A. E., Mechelen, I. V., Verbeke, G., Heitjan, D. F. and Meulders, M. (2005). Multiple imputation for model checking: Completed-data plots with missing and latent data. Biometrics 61 74–85.
  • Gelman, A., Meng, X.-L. and Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies (with discussion). Statist. Sinica 6 733–807.
  • Gelman, A. E. and Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences (with discussion). Statist. Sci. 7 457–472.
  • Ghosh, M. and Rao, J. N. K. (1994). Small area estimation: An appraisal. Statist. Sci. 9 55–76.
  • Gilks, W. R., Wang, C. C., Yvonnet, B. and Coursaget, P. (1993). Random-effects models for longitudinal data using Gibbs’ sampling. Biometrics 49 441–453.
  • Hartley, H. O. (1958). Maximum likelihood estimation from incomplete data. Biometrics 14 174–194.
  • Hartley, H. O. and Hocking, R. R. (1971). The analysis of incomplete data. Biometrics 27 783–808.
  • Heitjan, D. F. (1994). Ignorability in general complete-data models. Biometrika 81 701–708.
  • Heitjan, D. and Rubin, D. B. (1991). Ignorability and coarse data. Ann. Statist. 19 2244–2253.
  • Hsu, C. H., Taylor, J. M., Murray, S. and Commenges, D. (2006). Survival analysis using auxiliary variables via non-parametric multiple imputation. Stat. Med. 25 3503–3517.
  • Ibrahim, J. G., Chen, M.-H., Lipsitz, S. R. and Herring, A. H. (2005). Missing data methods in generalized linear models: A comparative review. J. Amer. Statist. Assoc. 100 332–346.
  • Ibrahim, J. G. and Molenberghs, G. (2009). Missing data methods in longitudinal studies: A review (with discussion). Test 18 1–43.
  • Kang, D. Y. and Schafer, J. L. (2007). Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Statist. Sci. 22 523–539.
  • Lange, K. (2004). Optimation. Springer, New York.
  • Littell, R. C., Milliken, G. A., Stroup, W. W. and Wolnger, R. D. (1996). SAS System for Mixed Models. SAS Institute Inc., Cary, NC.
  • Little, R. J. (1979). Maximum likelihood inference for multiple regression with missing values: A simulation study. J. Roy. Statist. Soc. Ser. B 41 76–87.
  • Little, R. J. A. (1986). Survey nonresponse adjustments for estimates of means. Internat. Statist. Rev. 54 139–157.
  • Little, R. J. A. (1988). Small sample inference about means from bivariate normal data with missing values. Comput. Statist. Data Anal. 7 161–178.
  • Little, R. J. A. (2006). Calibrated Bayes: A Bayes/frequentist roadmap. Amer. Statist. 60 213–223.
  • Little, R. J. A. and An, H. (2004). Robust likelihood-based analysis of multivariate data with missing values. Statist. Sinica 14 949–968.
  • Little, R. J. A. and Rubin, D. B. (1987). Statistical Analysis with Missing Data, 1st ed. Wiley, New York.
  • Little, R. J. A. and Rubin, D. B. (2002). Statistical Analysis with Missing Data, 2nd ed. Wiley, New York.
  • Long, Q., Little, R. J. and Lin, X. (2010). Estimating the CACE in trials involving multi-treatment arms subject to noncompliance: A Bayesian framework. J. Roy. Statist. Soc. Ser. C. To appear.
  • McKendrick, A. G. (1926). Applications of mathematics to medical problems. Proc. Edinburgh Math. Soc. 44 98–130.
  • McLachlan, G. J. and Krishnan, T. (1997). The EM Algorithm and Extensions. Wiley, New York.
  • Meng, X. L. and van Dyk, D. (1997). The EM algorithm—An old folk song sung to a fast new tune (with discussion). J. Roy. Statist. Soc. Ser. B 59 511–567.
  • MICE (2009). Multiple imputation via chained equations. Available at
  • Ngo, L. and Wand, M. P. (2004). Smoothing with mixed model software. J. Statist. Software V9 Issue 1.
  • Olkin, I. and Tate, R. F. (1961). Multivariate correlation models with mixed discrete and continuous variables. Ann. Math. Statist. 32 448–465.
  • Orchard, T. and Woodbury, M. A. (1972). A missing information principle: Theory and applications. In Proc. 6th Berkeley Symposium on Mathematical Statistics and Probabality 1 697–715. Univ. California Press, Berkeley, CA.
  • Peers, H. W. (1965). On confidence points and Bayesian probability points in the case of several parameters. J. Roy. Statist. Soc. Ser. B 27 9–16.
  • Pinheiro, J. C. and Bates, D. M. (2000). Mixed-Effects Models in S and S-PLUS. Springer, New York.
  • Raghunathan, T. E., Lepkowski, J. M., Van Hoewyk, J. and Solenberger, P. (2001). A multivariate technique for multiply imputing missing values using a sequence of regression models. Survey Methodology 27 85–95.
  • Raghunathan, T. E., Solenberger, P. W. and Van Hoewyk, J. (2009). IVEware: Imputation and variance estimation software. Available at
  • Robins, J. M., Rotnitzky, A. and Zhao, L. P. (1994). Estimation of regression coefficients when some regressors are not always observed. J. Amer. Statist. Assoc. 89 846–866.
  • Rosenbaum, P. R. and Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika 70 41–55.
  • Rotnitzky, A., Robins, J. M. and Scharfstein, D. O. (1998). Semiparametric regression for repeated measures outcomes with non-ignorable non-response. J. Amer. Statist. Assoc. 93 1321–1339.
  • Rubin, D. B. (1976). Inference and missing data (with discussion). Biometrika 63 581–592.
  • Rubin, D. B. (1977). Formalizing subjective notions about the effect of nonrespondents in sample surveys. J. Amer. Statist. Assoc. 72 538–543.
  • Rubin, D. B. (1978). Multiple imputations in sample surveys. In Proceedings of the Survey Research Methods Section 20–34. Amer. Statist. Assoc., Alexandria, VA.
  • Rubin, D. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Statist. 12 1151–1172.
  • Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. Wiley, New York.
  • Rubin, D. B. (1996). Multiple imputation after 18+ years. J. Amer. Statist. Assoc. 91 473–489.
  • Rubin, D. B. and Schenker, N. (1986). Multiple imputation for interval estimation from simple random samples with ignorable nonresponse. J. Amer. Statist. Assoc. 81 366–374.
  • Ruppert, D., Wand, M. P. and Carroll, R. J. (2003). Semiparametric Regression. Cambridge Univ. Press, Cambridge.
  • SAS (1992). The Mixed Procedure, Chapter 16 in SAS/STAT Software: Changes and Enhancements, Release 6.07. Technical Report P-229, SAS Institute, Inc., Cary, NC.
  • Schafer, J. L. (1997). Analysis of Incomplete Multivariate Data. CRC Press, New York.
  • Scharfstein, D., Rotnitsky, A. and Robins, J. (1999). Adjusting for nonignorable dropout using semiparametric models (with discussion). J. Amer. Statist. Assoc. 94 1096–1146.
  • Sundberg, R. (1974). Maximum likelihood theory for incomplete data from an exponential family. Scand. J. Statist. 1 49–58.
  • Tan, M. T. and Tian, G.-L. (2010). Bayesian Missing Data Problems: EM, Data Augmentation and Noniterative Computation. CRC Press, New York.
  • Tanner, M. A. and Wong, W. H. (1987). The calculation of posterior distributions by data augmentation (with discussion). J. Amer. Statist. Assoc. 82 528–550.
  • Van Buuren, S., Brand, J. P. L., Groothuis-Oudshoorn, K. and Rubin, D. B. (2006). Fully conditional specification in multivariate imputation. J. Statist. Comput. Simul. 76 1049–1064.
  • Welch, B. L. (1965). On comparisons between confidence point procedures in the case of a single parameter. J. Roy. Statist. Soc. Ser. B 27 1–8.
  • Zhang, G. and Little, R. J. (2009). Extensions of the penalized spline propensity prediction method of imputation. Biometrics 65 911–918. DOI: 10.1111/j.1541-0420.2008.01155.

See also

  • Discussion of: Calibrated Bayes, for Statistics in General, and Missing Data in Particular by R. J. A. Little.
  • Discussion of: Calibrated Bayes, for Statistics in General, and Missing Data in Particular by R. J. A. Little.
  • Rejoinder: Calibrated Bayes, for Statistics in General, and Missing Data in Particular.