The Annals of Statistics

On Bayes’s theorem for improper mixtures

Peter McCullagh and Han Han

Full-text: Open access


Although Bayes’s theorem demands a prior that is a probability distribution on the parameter space, the calculus associated with Bayes’s theorem sometimes generates sensible procedures from improper priors, Pitman’s estimator being a good example. However, improper priors may also lead to Bayes procedures that are paradoxical or otherwise unsatisfactory, prompting some authors to insist that all priors be proper. This paper begins with the observation that an improper measure on Θ satisfying Kingman’s countability condition is in fact a probability distribution on the power set. We show how to extend a model in such a way that the extended parameter space is the power set. Under an additional finiteness condition, which is needed for the existence of a sampling region, the conditions for Bayes’s theorem are satisfied by the extension. Lack of interference ensures that the posterior distribution in the extended space is compatible with the original parameter space. Provided that the key finiteness condition is satisfied, this probabilistic analysis of the extended model may be interpreted as a vindication of improper Bayes procedures derived from the original model.

Article information

Ann. Statist., Volume 39, Number 4 (2011), 2007-2020.

First available in Project Euclid: 24 August 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F15: Bayesian inference
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures

Countable measure lack of interference marginalization paradox


McCullagh, Peter; Han, Han. On Bayes’s theorem for improper mixtures. Ann. Statist. 39 (2011), no. 4, 2007--2020. doi:10.1214/11-AOS892.

Export citation


  • Akaike, H. (1980). The interpretation of improper prior distributions as limits of data dependent proper prior distribution. J. Roy. Statist. Soc. Ser. B 42 46–52.
  • Bernardo, J. M. and Smith, A. F. M. (1994). Bayesian Theory. Wiley, Chichester.
  • Dawid, A. P., Stone, M. and Zidek, J. V. (1973). Marginalization paradoxes in Bayesian and structural inference (with discussion). J. Roy. Statist. Soc. Ser. B 35 189–233.
  • Durrett, R. (2010). Probability: Theory and Examples. Cambridge Univ. Press, Cambridge.
  • Eaton, M. L. (1982). A method for evaluating improper prior distributions. In Statistical decision theory and related topics, III, Vol. 1 (West Lafayette, Ind., 1981) 329–352. Academic Press, New York.
  • Eaton, M. L. and Sudderth, W. D. (1995). The formal posterior of a standard flat prior in MANOVA is incoherent. Journal of the Italian Statistical Society 2 251–270.
  • Genest, C., McConway, K. J. and Schervish, M. (1986). Characterization of externally Bayesian pooling operators. Ann. Statist. 14 487–501.
  • Hartigan, J. A. (1983). Bayes Theory. Springer, New York.
  • Jaynes, E. T. (2003). Probability Theory: The Logic of Science. Cambridge Univ. Press, New York.
  • Kingman, J. F. C. (1993). Poisson Processes. Clarendon Press, Oxford.
  • Kotz, S. and Nadarajah, S. (2004). Multivariate t Distributions and Their Applications. Cambridge Univ. Press, Cambridge.
  • Lindley, D. V. (1973). Discussion of “Marginalization paradoxes in Bayesian and structural inference” by Dawid, Stone and Zidek. J. Roy. Statist. Soc. Ser. B 35 218–219.
  • Spiegelhalter, D. J. (1985). Exact Bayesian inference on the parameters of a Cauchy distribution with vague prior information. In Bayesian Statistics, 2 (Valencia, 1983) (J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith, eds.) 743–749. North-Holland, Amsterdam.
  • Stone, M. and Dawid, A. P. (1972). Un-Bayesian implications of improper Bayes inference in routine posterior statistical problems. Biometrika 59 369–375.
  • Taraldsen, G. and Lindqvist, B. H. (2010). Improper priors are not improper. Amer. Statist. 64 154–158.
  • Wallstrom, T. C. (2007). The marginalization paradox and the formal Bayes’ law. Bayesian Inference and Maximum Entropy Methods in Science and Engineering (K. Knuth et al., eds.). AIP Conference Proceedings 954 93–100. Saratoga Springs, New York.
  • Weerhandi, S. and Zidek, J. V. (1981). Multi-Bayesian statistical decision theory. J. Roy. Statist. Soc. Ser. A 144 85–93.