International Statistical Review

Statistical Decision Problems and Bayesian Nonparametric Methods

Eduardo Gutiérrez-Peña and Stephen G. Walker

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


This paper considers parametric statistical decision problems conducted within a Bayesian nonparametric context. Our work was motivated by the realisation that typical parametric model selection procedures are essentially incoherent. We argue that one solution to this problem is to use a flexible enough model in the first place, a model that will not be checked no matter what data arrive. Ideally, one would use a nonparametric model to describe all the uncertainty about the density function generating the data. However, parametric models are the preferred choice for many statisticians, despite the incoherence involved in model checking, incoherence that is quite often ignored for pragmatic reasons. In this paper we show how coherent parametric inference can be carried out via decision theory and Bayesian nonparametrics. None of the ingredients discussed here are new, but our main point only becomes evident when one sees all priors-even parametric ones-as measures on sets of densities as opposed to measures on finite-dimensional parameter spaces.

Article information

Internat. Statist. Rev., Volume 73, Number 3 (2005), 309-330.

First available in Project Euclid: 5 December 2005

Permanent link to this document

Zentralblatt MATH identifier

Coherence Consistency Decision theory Divergence Expected utility Nonparametric prior Parametric predictive density


Gutiérrez-Peña, Eduardo; Walker, Stephen G. Statistical Decision Problems and Bayesian Nonparametric Methods. Internat. Statist. Rev. 73 (2005), no. 3, 309--330.

Export citation


  • [1] Barron, A., Schervish, M.J. & Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Annals of Statistics, 27, 536-561.
  • [2] Bassetti, F., Bodini, A. & Regazzini, E. (2001). Outline of a new predictive and self-controlling methodology. Technical Report, IMATI, Milan. Available at
  • [3] Beran, R. (1977). Minimum Hellinger distance estimates for parametric models. Annals of Statistics, 5, 445-463.
  • [4] Bernardo, J.M. & Smith, A.F.M. (1994). \textit {Bayesian Theory}. Chichester: Wiley.
  • [5] Bernardo, J.M. & Rueda, R. (2002). Bayesian hypothesis testing: a reference approach. \textit { International Statistical Review}, 70, 351-372.
  • [6] Böhning, D. (1995). A review of reliable maximum likelihood algorithms for semiparametric mixture models. \textit {Journal of Statistical Planning and Inference}, 47, 5-28.
  • [7] Box, G.E.P. (1980). Sampling and Bayes inference in scientific modeling and robustness (with discussion). Journal of the Royal Statistical Society A, 143, 383-430.
  • [8] Csiszár, I. (1963). Eine informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizität von Markoffschen Ketten. Publ. Math. Inst. Hungar. Acad. Sci., 8, 85-107.
  • [9] Csiszár, I. (1967). Information-type measures of difference of probability distributions and indirect observations. Studia Sci. Math. Hungar., 2, 299-318.
  • [10] Dawid, A.P. (1982). The well-calibrated Bayesian. Journal of the American Statistical Association, 77, 605-610.
  • [11] de Finetti, B. (1937). La prévision: Ses lois logiques, ses sources subjectives. Annals of the Institute Henri Poincaré, 7, 1-68.
  • [12] DeGroot, M.H. (1970). Optimal Statistical Decisions. New York: McGraw-Hill.
  • [13] Denison, D.G.T., Holmes, C.C., Mallick, B.K. & Smith, A.F.M. (2002). Bayesian Methods for Nonlinear Classification and Regression. Chichester: Wiley.
  • [14] Dey, D., Sinha, D. & Müller, P. (1998). Practical Nonparametric and Semiparametric Bayesian Statistics. Lecture Notes in Statistics. New York: Springer.
  • [15] Draper, D. (1995). Assessment and propagation of model uncertainty (with discussion). \textit {Journal of the Royal Statistical Society B}, 57, 45-97.
  • [16] Draper, D. (1999). Discussion of the paper ''Bayesian nonparametric inference for random distributions and related functions'', by Walker, et al. \textit {Journal of the Royal Statistical Society B}, 61, 485-527.
  • [17] Escobar, M.D. & West, M. (1995). Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association, 90, 577-588.
  • [18] Everitt, B.S. & Hand, D.J. (1981). Finite Mixture Distributions. London: Chapman & Hall.
  • [19] Ferguson, T.S. (1973). A Bayesian analysis of some nonparametric problems. \textit {Annals of Statistics}, 1, 209-230.
  • [20] Gelfand, A.E. (1998). Approaches for semiparametric Bayesian regression. In Asymptotics, Nonparametrics and Time Series, Ed. S. Ghosh, pp. 615-629. New York: Marcel-Dekker.
  • [21] Gutiérrez-Peña, E. (1997). A Bayesian predictive semiparametric approach to variable selection and model comparison in regression. Bulletin of the International Statistical Institute, Tome LVII. Proceedings of the 51st Session of the ISI}, Istanbul, Turkey Invited Papers, Book 1, pp. 17-29.
  • [22] Gutiérrez-Peña, E. & Walker, S.G. (2001). A Bayesian Predictive Approach to Model Selection. Journal of Statistical Planning and Inference, 93, 259-276.
  • [23] Haussler, D. & Opper, M. (1997). Mutual information, metric entropy and cumulative relative entropy risk. Annals of Statistics, 25, 2451-2492.
  • [24] Hirshleifer, J. & Riley, J.G. (1992). The Analysis of Uncertainty and Information. Cambridge: Cambridge University Press.
  • [25] Lavine, M. (1992). Some aspects of Pólya tree distributions for statistical modelling. Annals of Statistics, 20, 1203-1221.
  • [26] Lavine, M. (1994). More aspects of Pólya trees for statistical modelling. Annals of Statistics, 22, 1161-1176.
  • [27] Lindley, D.V. (1957) A statistical paradox. Biometrika, 44, 187-192.
  • [28] Lindley, D.V. (1978). The Bayesian Approach (with discussion). Scandinavian Journal of Statistics, 5, 1-26.
  • [29] Lindley, D.V. (1997). The choice of sample size. The Statistician, 46, 129-138.
  • [30] Lindsey, J.K. (1999). Some statistical heresies. \textit {The Statistician}, 48, 1-40.
  • [31] Lo, A.Y. (1984). On a class of Bayesian nonparametric estimates I. Density estimates. Annals of Statistics, 12, 351-357.
  • [32] MacEachern, S.N. & Müller, P. (1998). Estimating mixtures of Dirichlet process models. Journal of Computational and Graphical Statistics, 7, 223-238.
  • [33] O'Hagan, A. (1978). Curve Fitting and Optimal Design for Prediction. Journal of the Royal Statistical Society B, 40, 1-42.
  • [34] Parr, W.C. & Schucany, W.R. (1980). Minimum distance and robust estimation. Journal of the American Statistical Association, 75, 616-624.%
  • [35] Proschan, F. (1963). Theoretical explanation of observed decreasing failure rate. Technometrics, 5, 375-383.
  • [36] Rubin, D.B. (1981). The Bayesian bootstrap. \textit {Annals of Statistics}, 9, 130-134.
  • [37] Titterington, D.M., Smith, A.F.M. & Makov, U.E. (1985). Statistical Analysis of Finite Mixture Distributions. Chichester: Wiley.
  • [38] Walker, S.G., Damien P., Laud, P.W. & Smith, A.F.M. (1999). Bayesian nonparametric inference for random distributions and related functions (with discussion). Journal of the Royal Statistical Society Series B, 61, 485-527.
  • [39] Walker, S.G. (2003a). How many samples?: a Bayesian nonparametric approach. The Statistician, 52, 475-482.
  • [40] Walker, S.G. (2003b). On sufficient conditions for Bayesian consistency Biometrika, 90, 482-488.
  • [41] Walker, S.G. & Gutiérrez-Peña, E. (1999). Robustifying Bayesian Procedures (with discussion). Bayesian Statistics 6, Eds. J.M. Bernardo, J.O. Berger, A.P. Dawid, A.F.M. Smith, pp. 685-710. Oxford: Oxford University Press.
  • [42] Walker, S.G. & Gutiérrez-Peña, E. (2005). Bayesian Parametric Inference in a Nonparametric Framework. To appear in Test.
  • [43] Walker, S.G., Gutiérrez-Peña, E. & Muliere, P. (2001). A Decision Theoretic Approach to Model Averaging. The Statistician, 50, 31-39.