Bayesian Analysis

On Bayesian Oracle Properties

Wenxin Jiang and Cheng Li

Full-text: Open access

Abstract

When model uncertainty is handled by Bayesian model averaging (BMA) or Bayesian model selection (BMS), the posterior distribution possesses a desirable “oracle property” for parametric inference, if for large enough data it is nearly as good as the oracle posterior, obtained by assuming unrealistically that the true model is known and only the true model is used. We study the oracle properties in a very general context of quasi-posterior, which can accommodate non-regular models with cubic root asymptotics and partial identification. Our approach for proving the oracle properties is based on a unified treatment that bounds the posterior probability of model mis-selection. This theoretical framework can be of interest to Bayesian statisticians who would like to theoretically justify their new model selection or model averaging methods in addition to empirical results. Furthermore, for non-regular models, we obtain nontrivial conclusions on the choice of prior penalty on model complexity, the temperature parameter of the quasi-posterior, and the advantage of BMA over BMS.

Article information

Source
Bayesian Anal., Volume 14, Number 1 (2019), 235-260.

Dates
First available in Project Euclid: 16 May 2018

Permanent link to this document
https://projecteuclid.org/euclid.ba/1526457619

Digital Object Identifier
doi:10.1214/18-BA1097

Mathematical Reviews number (MathSciNet)
MR3910045

Zentralblatt MATH identifier
07001982

Subjects
Primary: 62E99: None of the above, but in this section 62F15: Bayesian inference

Keywords
Bayesian model selection consistency model averaging oracle property cubic root asymptotics partial identification

Rights
Creative Commons Attribution 4.0 International License.

Citation

Jiang, Wenxin; Li, Cheng. On Bayesian Oracle Properties. Bayesian Anal. 14 (2019), no. 1, 235--260. doi:10.1214/18-BA1097. https://projecteuclid.org/euclid.ba/1526457619


Export citation

References

  • Bissiri, P. G., Holmes, C. C., and Walker, S. G. (2016). “A general framework for updating belief distributions.” Journal of Royal Statistical Society: Series B, 78: 1103–1130.
  • Castillo, I., Schmidt-Hieber, J., and van der Vaart, A. (2015). “Bayesian linear regression with sparse priors.” The Annals of Statistics, 43: 1986–2018.
  • Chen, X., Christensen, T., and Tamer, E. (2016). “MCMC confidence sets for identified sets.” Cowles Foundation Discussion Paper, No. 2037. ArXiv:1605.00499.
  • Chernozhukov, V. and Hong, H. (2003). “An MCMC approach to classical estimation.” Journal of Econometrics, 115: 293–346.
  • Chernozhukov, V., Hong, H., and Tamer, E. (2007). “Estimation and confidence regions for parameter sets in econometric models.” Econometrica, 75: 1243–1284.
  • Drton, M. and Plummer, M. (2017). “A Bayesian information criterion for singular models.” Journal of Royal Statistical Society: Series B, 79: 323–380.
  • Fan, J. and Li, R. (2001). “Variable selection via nonconcave penalized likelihood and its oracle properties.” Journal of the American Statistical Association, 96: 1348–1360.
  • Grünwald, P. and van Ommen, T. (2014). “Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it.” ArXiv:1412.3730.
  • Gustafson, P. (2015). Bayesian Inference for Partially Identified Models: Exploring the Limits of Limited Data. CRC Press, New York.
  • Hong, H. and Preston, B. (2012). “Bayesian averaging, prediction and nonnested model selection.” Journal of Econometrics, 167: 358–369.
  • Ishwaran, H. and Rao, J. S. (2011). “Consistency of spike and slab regression.” Statistics and Probability Letters, 81: 1920–1928.
  • Jiang, W. and Li, C. (2015). “A note on Bayesian oracle properties.” ArXiv: 1507.05723v1.
  • Jiang, W. and Li, C. (2019). “Supplement to “On Bayesian Oracle Properties”.” Bayesian Analysis.
  • Johnson, V. E. and Rossell, D. (2012). “Bayesian model selection in high-dimensional settings.” Journal of the American Statistical Association, 107: 649–660.
  • Jun, S. J., Pinkse, J., and Wan, Y. (2011). “Root-n consistent robust integration-based estimation.” Journal of Multivariate Analysis, 102: 828–846.
  • Jun, S. J., Pinkse, J., and Wan, Y. (2015). “Classical Laplace estimation for cube-root-n consistent estimators: Improved convergence rates and rate-adaptive inference.” Journal of Econometrics, 187: 201–216.
  • Kim, J. and Pollard, D. (1990). “Cubic root asymptotics.” The Annals of Statistics, 18: 191–219.
  • Kline, B. and Tamer, E. (2016). “Bayesian inference in a class of partially identified models.” Quantitative Economics, 7: 329–366.
  • Li, C. and Jiang, W. (2016). “On oracle property and asymptotic validity of Bayesian generalized method of moments.” Journal of Multivariate Analysis, 145: 132–147.
  • Li, C., Jiang, W., and Tanner, M. A. (2014). “General inequalities for Gibbs posterior with nonadditive empirical risk.” Econometric Theory, 30: 1247–1271.
  • Liang, F., Song, Q., and Yu, K. (2013). “Bayesian subset modeling for high-dimensional generalized linear models.” Journal of the American Statistical Association, 108: 589–606.
  • Manski, C. (1975). “Maximum score estimation of the stochastic utility model of choice.” Journal of Econometrics, 3: 205–228.
  • Moon, H. R. and Schorfheide, F. (2012). “Bayesian and frequentist inference in partially identified models.” Econometrica, 80: 755–782.
  • Newey, W. K. (1991). “Uniform convergence in probability and stochastic equicontinuity.” Econometrica, 59: 1161–1167.
  • Poirier, D. J. (1998). “Revising beliefs in nonidentified models.” Econometric Theory, 14: 483–509.
  • Schwartz, G. (1978). “Estimating the dimension of a model.” The Annals of Statistics, 6: 461–464.
  • Shen, X. (2002). “Asymptotic normality of semiparametric and nonparametric posterior distributions.” Journal of the American Statistical Association, 97: 222–235.
  • Syring, N. and Martin, R. (2017). “Gibbs posterior inference on the minimum clinically important difference.” Journal of Statistical Planning and Inference, 187: 67–77.
  • van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge University Press.
  • Wan, Y. (2013). “An integration-based approach to moment inequality models.” Manuscript. University of Toronto. http://individual.utoronto.ca/yuanyuanwan/Integration_Wan.pdf.
  • Wasserman, L. (2000). “Bayesian model selection and model averaging.” Journal of Mathematical Psychology, 44: 92–107.

Supplemental materials

  • Supplement to “On Bayesian Oracle Properties”. We provide the technical proofs of Theorem 3, Theorem 4, and Corollary 1, as well as additional discussion on the Bayesian oracle properties for partial identification.