The Annals of Statistics

The problem of regions

Bradley Efron and Robert Tibshirani

Full-text: Open access


In the problem of regions, we wish to know which one of a discrete set of possibilities applies to a continuous parameter vector. This problem arises in the following way: we compute a descriptive statistic from a set of data, notice an interesting feature and wish to assign a confidence level to that feature. For example, we compute a density estimate and notice that the estimate is bimodal. What confidence can we assign to bimodality? A natural way to measure confidence is via the bootstrap: we compute our descriptive statistic on a large number of bootstrap data sets and record the proportion of times that the feature appears. This seems like a plausible measure of confidence for the feature. The paper studies the construction of such confidence values and examines to what extent they approximate frequentist $p$-values and Bayesian a posteriori probabilities. We derive more accurate confidence levels using both frequentist and objective Bayesian approaches. The methods are illustrated with a number of examples, including polynomial model selection and estimating the number of modes of a density.

Article information

Ann. Statist. Volume 26, Number 5 (1998), 1687-1718.

First available in Project Euclid: 21 June 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: Primary 62G10
Secondary: secondary 62G09

Discrete estimation problems objective Bayes methods bootstrap reweighting metric-free methods


Efron, Bradley; Tibshirani, Robert. The problem of regions. Ann. Statist. 26 (1998), no. 5, 1687--1718. doi:10.1214/aos/1024691353.

Export citation


  • Beran, R. (1987). Prepivoting to reduce level error of confidence sets. Biometrika 74 457-468.
  • DiCiccio, T. and Efron, B. (1992). More accurate confidence limits in exponential families. Biometrika 79 231-245.
  • Efron, B. (1982). The jackknife, the bootstrap and other resampling plans. SIAM, Philadelphia.
  • Efron, B. (1985). Bootstrap confidence intervals for a class of parametric problems. Biometrika 72 45-58.
  • Efron, B. (1987). Better bootstrap confidence intervals (with discussion). J. Amer. Statist. Assoc. 82 171-200.
  • Efron, B. and Feldman, D. (1991). Compliance as an explanatory variable in clinical trials, J. Amer. Statist. Assoc. 86 9-26.
  • Efron, B., Halloran, E. and Holmes, S. (1996). Bootstrap confidence levels for phy logenetic trees. Proc. Nat. Acad. Sci. U.S.A. 93 13429-13434.
  • Efron, B. and Tibshirani, R. (1993). An Introduction to the Bootstrap. Chapman and Hall, London.
  • Efron, B. and Tibshirani, R. (1996). The problem of regions. Stanford Technical Report 192. Available at
  • Felsenstein, J. (1985). Confidence limits on phy logenies: an approach using the bootstrap. Evolution 783-791.
  • Hall, P. (1992). The Bootstrap and Edgeworld Expansion. Springer, New York.
  • Izenman, A. and Sommer, L. (1988). Philatelic mixtures and multimodal densities. J. Amer. Statist. Assoc. 83 941-953.
  • Loh, W. Y. (1987). Calibrating confidence coefficients. J. Amer. Statist. Assoc. 82 152-162.
  • Mallows, C. (1973). Some comments on cp. Technometrics 15 661-675.
  • Rubin, D. (1981). The bay esian bootstrap. Ann. Statist. 9 130-134.
  • Tibshirani, R. (1989). Non-informative priors for one parameter of many. Biometrika 76 604-608.
  • Welch, B. and Peers, H. (1963). On formulae for confidence points based on intervals of weighted likelihoods. J. Roy. Statist. Soc. Ser. B 25 318-329.
  • Weng, C. S. (1989). On a second-order asy mptotic property of the Bayesian bootstrap mean. Ann. Statist. 17 705-710.