Statistical Science

Dempster–Shafer Theory and Statistical Inference with Weak Beliefs

Ryan Martin, Jianchun Zhang, and Chuanhai Liu
Source: Statist. Sci. Volume 25, Number 1 (2010), 72-87.

Abstract

The Dempster–Shafer (DS) theory is a powerful tool for probabilistic reasoning based on a formal calculus for combining evidence. DS theory has been widely used in computer science and engineering applications, but has yet to reach the statistical mainstream, perhaps because the DS belief functions do not satisfy long-run frequency properties. Recently, two of the authors proposed an extension of DS, called the weak belief (WB) approach, that can incorporate desirable frequency properties into the DS framework by systematically enlarging the focal elements. The present paper reviews and extends this WB approach. We present a general description of WB in the context of inferential models, its interplay with the DS calculus, and the maximal belief solution. New applications of the WB method in two high-dimensional hypothesis testing problems are given. Simulations show that the WB procedures, suitably calibrated, perform well compared to popular classical methods. Most importantly, the WB approach combines the probabilistic reasoning of DS with the desirable frequency properties of classical statistics.

First Page: Show Hide
Full-text: Access denied (no subscription detected)
We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber.
If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text
Links and Identifiers

Permanent link to this document: http://projecteuclid.org/euclid.ss/1280841734
Digital Object Identifier: doi:10.1214/10-STS322
Mathematical Reviews number (MathSciNet): MR2741815
Zentralblatt MATH identifier: 05945286

References

[1] Dempster, A. P. (1963). Further examples of inconsistencies in the fiducial argument. Ann. Math. Statist. 34 884–891.
Mathematical Reviews (MathSciNet): MR150865
Zentralblatt MATH: 0214.17902
Digital Object Identifier: doi:10.1214/aoms/1177704011
Project Euclid: euclid.aoms/1177704011
[2] Dempster, A. P. (1966). New methods for reasoning towards posterior distributions based on sample data. Ann. Math. Statist. 37 355–374.
Mathematical Reviews (MathSciNet): MR187357
Zentralblatt MATH: 0178.54302
Digital Object Identifier: doi:10.1214/aoms/1177699517
Project Euclid: euclid.aoms/1177699517
[3] Dempster, A. P. (1967). Upper and lower probabilities induced by a multivalued mapping. Ann. Math. Statist. 38 325–339.
Mathematical Reviews (MathSciNet): MR207001
Zentralblatt MATH: 0168.17501
Digital Object Identifier: doi:10.1214/aoms/1177698950
Project Euclid: euclid.aoms/1177698950
[4] Dempster, A. P. (1968). A generalization of Bayesian inference (with discussion). J. Roy. Statist. Soc. Ser. B 30 205–247.
Mathematical Reviews (MathSciNet): MR238428
[5] Dempster, A. P. (1969). Upper and lower probability inferences for families of hypotheses with monotone density ratios. Ann. Math. Statist. 40 953–969.
Mathematical Reviews (MathSciNet): MR246427
Zentralblatt MATH: 0211.50204
Digital Object Identifier: doi:10.1214/aoms/1177697600
Project Euclid: euclid.aoms/1177697600
[6] Dempster, A. P. (2008). Dempster–Shafer calculus for statisticians. Internat. J. Approx. Reason. 48 265–277.
Mathematical Reviews (MathSciNet): MR2419025
Zentralblatt MATH: 05683169
Digital Object Identifier: doi:10.1016/j.ijar.2007.03.004
[7] Denoeux, T. (2006). Constructing belief functions from sample data using multinomial confidence regions. Internat. J. Approx. Reason. 42 228–252.
Mathematical Reviews (MathSciNet): MR2246874
Zentralblatt MATH: 1100.68112
Digital Object Identifier: doi:10.1016/j.ijar.2006.01.001
[8] Edlefsen, P. T., Liu, C. and Dempster, A. P. (2009). Estimating limits from Poisson counting data using Dempster–Shafer analysis. Ann. Appl. Statist. 3 764–790.
[9] Fisher, R. A. (1930). Inverse probability. Proceedings of the Cambridge Philosophical Society 26 528–535.
[10] Fisher, R. A. (1935). The logic of inductive inference. J. Roy. Statist. Soc. 98 39–82.
[11] Fraser, D. A. S. (1968). The Structure of Inference. Wiley, New York.
Mathematical Reviews (MathSciNet): MR235643
Zentralblatt MATH: 0164.48703
[12] Hannig, J. (2009). On generalized fiducial inference. Statist. Sinica 19 491–544.
Mathematical Reviews (MathSciNet): MR2514173
Zentralblatt MATH: 1168.62004
Digital Object Identifier: doi:10.1007/s10114-008-6010-1
[13] Kohlas, J. and Monney, P.-A. (2008). An algebraic theory for statistical information based on the theory of hints. Internat. J. Approx. Reason. 48 378–398.
Mathematical Reviews (MathSciNet): MR2419014
Zentralblatt MATH: 05683170
Digital Object Identifier: doi:10.1016/j.ijar.2007.05.003
[14] Kushner, H. J. and Yin, G. G. (2003). Stochastic Approximation and Recursive Algorithms and Applications, 2nd ed. Springer, New York.
Mathematical Reviews (MathSciNet): MR1993642
Zentralblatt MATH: 1026.62084
[15] Lindley, D. V. (1958). Fiducial distributions and Bayes’ theorem. J. Roy. Statist. Soc. Ser. B 20 102–107.
Mathematical Reviews (MathSciNet): MR95550
[16] Martin, R. and Ghosh, J. K. (2008). Stochastic approximation and Newton’s estimate of a mixing distribution. Statist. Sci. 23 365–382.
Mathematical Reviews (MathSciNet): MR2483909
Digital Object Identifier: doi:10.1214/08-STS265
Project Euclid: euclid.ss/1233153064
[17] Robbins, H. and Monro, S. (1951). A stochastic approximation method. Ann. Math. Statist. 22 400–407.
Mathematical Reviews (MathSciNet): MR42668
Zentralblatt MATH: 0054.05901
Digital Object Identifier: doi:10.1214/aoms/1177729586
Project Euclid: euclid.aoms/1177729586
[18] Shafer, G. (1976). A Mathematical Theory of Evidence. Princeton Univ. Press, Princeton, NJ.
Mathematical Reviews (MathSciNet): MR464340
Zentralblatt MATH: 0359.62002
[19] Shafer, G. (1978/79). Nonadditive probabilities in the work of Bernoulli and Lambert. Arch. Hist. Exact Sci. 19 309–370.
Mathematical Reviews (MathSciNet): MR515919
Digital Object Identifier: doi:10.1007/BF00330065
[20] Shafer, G. (1979). Allocations of probability. Ann. Probab. 7 827–839.
Mathematical Reviews (MathSciNet): MR542132
Zentralblatt MATH: 0414.60002
Digital Object Identifier: doi:10.1214/aop/1176994941
Project Euclid: euclid.aop/1176994941
[21] Shafer, G. (1981). Constructive probability. Synthese 48 1–60.
Mathematical Reviews (MathSciNet): MR623413
Digital Object Identifier: doi:10.1007/BF01064627
[22] Shafer, G. (1982). Belief functions and parametric models (with discussion). J. Roy. Statist. Soc. Ser. B 44 322–352.
Mathematical Reviews (MathSciNet): MR693232
[23] Yager, R. and Liu, L. (eds.) (2008). Classic Works of the Dempster–Shafer Theory of Belief Functions. Stud. Fuzziness Soft Comput. 219. Springer, Berlin.
Mathematical Reviews (MathSciNet): MR2458525
[24] Zabell, S. L. (1992). R. A. Fisher and the fiducial argument. Statist. Sci. 7 369–387.
Mathematical Reviews (MathSciNet): MR1181418
Digital Object Identifier: doi:10.1214/ss/1177011233
Project Euclid: euclid.ss/1177011233
[25] Zhang, J. and Liu, C. (2010). Dempster–Shafer inference with weak beliefs. Statistica Sinica. To appear.

2012 © Institute of Mathematical Statistics

Statistical Science

Statistical Science