To the frequentist who computes posteriors, not all priors are useful asymptotically: in this paper, a Bayesian perspective on test sequences is proposed and Schwartz’s Kullback–Leibler condition is generalised to widen the range of frequentist applications of posterior convergence. With Bayesian tests and a weakened form of contiguity termed remote contiguity, we prove simple and fully general frequentist theorems, for posterior consistency and rates of convergence, for consistency of posterior odds in model selection, and for conversion of sequences of credible sets into sequences of confidence sets with asymptotic coverage one. For frequentist uncertainty quantification, this means that a prior inducing remote contiguity allows one to enlarge credible sets of calculated, simulated or approximated posteriors to obtain asymptotically consistent confidence sets.
References
[1] Abbe, E., Bandeira, A. S. and Hall, G. (2016). Exact recovery in the stochastic block model. IEEE Trans. Inf. Theory 62 471–487. 1359.94047 10.1109/TIT.2015.2490670[1] Abbe, E., Bandeira, A. S. and Hall, G. (2016). Exact recovery in the stochastic block model. IEEE Trans. Inf. Theory 62 471–487. 1359.94047 10.1109/TIT.2015.2490670
[2] Barron, A. (1988). The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions. Technical Report 7, Dept. Statistics, Univ. Illinois.[2] Barron, A. (1988). The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions. Technical Report 7, Dept. Statistics, Univ. Illinois.
[3] Barron, A., Schervish, M. J. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536–561. 0980.62039 10.1214/aos/1017939142 euclid.aos/1018031206[3] Barron, A., Schervish, M. J. and Wasserman, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536–561. 0980.62039 10.1214/aos/1017939142 euclid.aos/1018031206
[4] Bayarri, M. J. and Berger, J. O. (2004). The interplay of Bayesian and frequentist analysis. Statist. Sci. 19 58–80. 1062.62001 10.1214/088342304000000116 euclid.ss/1089808273[4] Bayarri, M. J. and Berger, J. O. (2004). The interplay of Bayesian and frequentist analysis. Statist. Sci. 19 58–80. 1062.62001 10.1214/088342304000000116 euclid.ss/1089808273
[8] Breiman, L., Le Cam, L. and Schwartz, L. (1964). Consistent estimates and zero–one sets. Ann. Math. Stat. 35 157–161. 0223.62029 10.1214/aoms/1177703737 euclid.aoms/1177703737[8] Breiman, L., Le Cam, L. and Schwartz, L. (1964). Consistent estimates and zero–one sets. Ann. Math. Stat. 35 157–161. 0223.62029 10.1214/aoms/1177703737 euclid.aoms/1177703737
[9] Bühlmann, P. and van de Geer, S. (2011). Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer Series in Statistics. Springer, Heidelberg.[9] Bühlmann, P. and van de Geer, S. (2011). Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer Series in Statistics. Springer, Heidelberg.
[10] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903–923. 0778.62003 10.1214/aos/1176349157 euclid.aos/1176349157[10] Cox, D. D. (1993). An analysis of Bayesian inference for nonparametric regression. Ann. Statist. 21 903–923. 0778.62003 10.1214/aos/1176349157 euclid.aos/1176349157
[11] Doob, J. L. (1949). Application of the theory of martingales. In Le Calcul des Probabilités et Ses Applications. Colloques Internationaux du Centre National de la Recherche Scientifique 13 23–27. Centre National de la Recherche Scientifique, Paris.[11] Doob, J. L. (1949). Application of the theory of martingales. In Le Calcul des Probabilités et Ses Applications. Colloques Internationaux du Centre National de la Recherche Scientifique 13 23–27. Centre National de la Recherche Scientifique, Paris.
[12] Falk, M., Padoan, S. and Rizzelli, S. (2019). Strong convergence of multivariate maxima. Preprint. Available at arXiv:1903.10596. 1903.10596 07199536 10.1017/jpr.2019.100[12] Falk, M., Padoan, S. and Rizzelli, S. (2019). Strong convergence of multivariate maxima. Preprint. Available at arXiv:1903.10596. 1903.10596 07199536 10.1017/jpr.2019.100
[13] Freedman, D. (1999). On the Bernstein–von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119–1140. 0957.62002 euclid.aos/1017938917[13] Freedman, D. (1999). On the Bernstein–von Mises theorem with infinite-dimensional parameters. Ann. Statist. 27 1119–1140. 0957.62002 euclid.aos/1017938917
[14] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531. 1105.62315 10.1214/aos/1016218228 euclid.aos/1016218228[14] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531. 1105.62315 10.1214/aos/1016218228 euclid.aos/1016218228
[15] Ghosal, S. and van der Vaart, A. (2017). Fundamentals of Nonparametric Bayesian Inference. Cambridge Series in Statistical and Probabilistic Mathematics 44. Cambridge Univ. Press, Cambridge. 1376.62004[15] Ghosal, S. and van der Vaart, A. (2017). Fundamentals of Nonparametric Bayesian Inference. Cambridge Series in Statistical and Probabilistic Mathematics 44. Cambridge Univ. Press, Cambridge. 1376.62004
[16] Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122–1170. 1183.62062 10.1214/09-AOS738 euclid.aos/1266586625[16] Giné, E. and Nickl, R. (2010). Confidence bands in density estimation. Ann. Statist. 38 1122–1170. 1183.62062 10.1214/09-AOS738 euclid.aos/1266586625
[17] Greenwood, P. E. and Shiryayev, A. N. (1985). Contiguity and the Statistical Invariance Principle. Stochastics Monographs 1. Gordon & Breach, New York. 0659.62029[17] Greenwood, P. E. and Shiryayev, A. N. (1985). Contiguity and the Statistical Invariance Principle. Stochastics Monographs 1. Gordon & Breach, New York. 0659.62029
[18] Hengartner, N. W. and Stark, P. B. (1995). Finite-sample confidence envelopes for shape-restricted densities. Ann. Statist. 23 525–550. 0828.62043 10.1214/aos/1176324534 euclid.aos/1176324534[18] Hengartner, N. W. and Stark, P. B. (1995). Finite-sample confidence envelopes for shape-restricted densities. Ann. Statist. 23 525–550. 0828.62043 10.1214/aos/1176324534 euclid.aos/1176324534
[19] Johnson, V. E. and Rossell, D. (2010). On the use of non-local prior densities in Bayesian hypothesis tests. J. R. Stat. Soc. Ser. B. Stat. Methodol. 72 143–170. 1411.62019 10.1111/j.1467-9868.2009.00730.x[19] Johnson, V. E. and Rossell, D. (2010). On the use of non-local prior densities in Bayesian hypothesis tests. J. R. Stat. Soc. Ser. B. Stat. Methodol. 72 143–170. 1411.62019 10.1111/j.1467-9868.2009.00730.x
[21] Kleijn, B. and van Waaij, J. (2018). Recovery and confidence sets of communities in a sparse stochastic block model. Preprint. Available at arXiv:1810.09533. 1810.09533[21] Kleijn, B. and van Waaij, J. (2018). Recovery and confidence sets of communities in a sparse stochastic block model. Preprint. Available at arXiv:1810.09533. 1810.09533
[22] Kleijn, B. J. K. (2021). Supplement to “Frequentist validity of Bayesian limits.” https://doi.org/10.1214/20-AOS1952SUPP[22] Kleijn, B. J. K. (2021). Supplement to “Frequentist validity of Bayesian limits.” https://doi.org/10.1214/20-AOS1952SUPP
[23] Kleijn, B. J. K. and Zhao, Y. Y. (2019). Criteria for posterior consistency and convergence at a rate. Electron. J. Stat. 13 4709–4742. 1435.62110 10.1214/19-EJS1633[23] Kleijn, B. J. K. and Zhao, Y. Y. (2019). Criteria for posterior consistency and convergence at a rate. Electron. J. Stat. 13 4709–4742. 1435.62110 10.1214/19-EJS1633
[24] Le Cam, L. (1960). Locally asymptotically normal families of distributions. Certain approximations to families of distributions and their use in the theory of estimation and testing hypotheses. Univ. Calif. Publ. Statist. 3 37–98. 0104.12701[24] Le Cam, L. (1960). Locally asymptotically normal families of distributions. Certain approximations to families of distributions and their use in the theory of estimation and testing hypotheses. Univ. Calif. Publ. Statist. 3 37–98. 0104.12701
[25] Le Cam, L. (1972). Limits of experiments. In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability (Univ. California, Berkeley, Calif., 1970/1971), Vol. I: Theory of Statistics 245–261. Univ. California Press, Berkeley, CA. MR0415819 0271.62004[25] Le Cam, L. (1972). Limits of experiments. In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability (Univ. California, Berkeley, Calif., 1970/1971), Vol. I: Theory of Statistics 245–261. Univ. California Press, Berkeley, CA. MR0415819 0271.62004
[27] Le Cam, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer Series in Statistics. Springer, New York. 0605.62002[27] Le Cam, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer Series in Statistics. Springer, New York. 0605.62002
[28] Le Cam, L. and Yang, G. L. (1988). On the preservation of local asymptotic normality under information loss. Ann. Statist. 16 483–520. 0661.62016 10.1214/aos/1176350817 euclid.aos/1176350817[28] Le Cam, L. and Yang, G. L. (1988). On the preservation of local asymptotic normality under information loss. Ann. Statist. 16 483–520. 0661.62016 10.1214/aos/1176350817 euclid.aos/1176350817
[29] Le Cam, L. and Yang, G. L. (1990). Asymptotics in Statistics: Some Basic Concepts. Springer Series in Statistics. Springer, New York. 0719.62003[29] Le Cam, L. and Yang, G. L. (1990). Asymptotics in Statistics: Some Basic Concepts. Springer Series in Statistics. Springer, New York. 0719.62003
[31] Low, M. G. (1997). On nonparametric confidence intervals. Ann. Statist. 25 2547–2554. 0894.62055 10.1214/aos/1030741084 euclid.aos/1030741084[31] Low, M. G. (1997). On nonparametric confidence intervals. Ann. Statist. 25 2547–2554. 0894.62055 10.1214/aos/1030741084 euclid.aos/1030741084
[32] Mossel, E., Neeman, J. and Sly, A. (2016). Consistency thresholds for the planted bisection model. Electron. J. Probab. 21 Art. ID 21. 1336.05117 10.1214/16-EJP4185[32] Mossel, E., Neeman, J. and Sly, A. (2016). Consistency thresholds for the planted bisection model. Electron. J. Probab. 21 Art. ID 21. 1336.05117 10.1214/16-EJP4185
[33] Padoan, S. and Rizzelli, S. (2019). Strong consistency of nonparametric Bayesian inferential methods for multivariate max-stable distributions. Preprint. Available at arXiv:1904.00245. 1904.00245[33] Padoan, S. and Rizzelli, S. (2019). Strong consistency of nonparametric Bayesian inferential methods for multivariate max-stable distributions. Preprint. Available at arXiv:1904.00245. 1904.00245
[34] Schwartz, L. (1965). On Bayes procedures. Z. Wahrsch. Verw. Gebiete 4 10–26. 0158.17606 10.1007/BF00535479[34] Schwartz, L. (1965). On Bayes procedures. Z. Wahrsch. Verw. Gebiete 4 10–26. 0158.17606 10.1007/BF00535479
[35] Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687–714. 1041.62022 10.1214/aos/1009210686 euclid.aos/1009210686[35] Shen, X. and Wasserman, L. (2001). Rates of convergence of posterior distributions. Ann. Statist. 29 687–714. 1041.62022 10.1214/aos/1009210686 euclid.aos/1009210686
[36] Szabó, B., van der Vaart, A. W. and van Zanten, J. H. (2015). Frequentist coverage of adaptive nonparametric Bayesian credible sets. Ann. Statist. 43 1391–1428. 1317.62040 10.1214/14-AOS1270 euclid.aos/1434546205[36] Szabó, B., van der Vaart, A. W. and van Zanten, J. H. (2015). Frequentist coverage of adaptive nonparametric Bayesian credible sets. Ann. Statist. 43 1391–1428. 1317.62040 10.1214/14-AOS1270 euclid.aos/1434546205
[37] Taylor, J. and Tibshirani, R. J. (2015). Statistical learning and selective inference. Proc. Natl. Acad. Sci. USA 112 7629–7634. 1359.62228 10.1073/pnas.1507583112[37] Taylor, J. and Tibshirani, R. J. (2015). Statistical learning and selective inference. Proc. Natl. Acad. Sci. USA 112 7629–7634. 1359.62228 10.1073/pnas.1507583112
[38] Tokdar, S. T., Zhu, Y. M. and Ghosh, J. K. (2010). Bayesian density regression with logistic Gaussian process and subspace projection. Bayesian Anal. 5 319–344. 1330.62182 10.1214/10-BA605[38] Tokdar, S. T., Zhu, Y. M. and Ghosh, J. K. (2010). Bayesian density regression with logistic Gaussian process and subspace projection. Bayesian Anal. 5 319–344. 1330.62182 10.1214/10-BA605
[39] Torgersen, E. (1991). Comparison of Statistical Experiments. Encyclopedia of Mathematics and Its Applications 36. Cambridge Univ. Press, Cambridge. 0732.62009[39] Torgersen, E. (1991). Comparison of Statistical Experiments. Encyclopedia of Mathematics and Its Applications 36. Cambridge Univ. Press, Cambridge. 0732.62009
[40] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer Series in Statistics. Springer, New York. 0862.60002[40] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer Series in Statistics. Springer, New York. 0862.60002
[41] Walker, S. (2004). New approaches to Bayesian consistency. Ann. Statist. 32 2028–2043. 1056.62040 10.1214/009053604000000409 euclid.aos/1098883780[41] Walker, S. (2004). New approaches to Bayesian consistency. Ann. Statist. 32 2028–2043. 1056.62040 10.1214/009053604000000409 euclid.aos/1098883780
[42] Walker, S. G., Lijoi, A. and Prünster, I. (2007). On rates of convergence for posterior distributions in infinite-dimensional models. Ann. Statist. 35 738–746. 1117.62047 10.1214/009053606000001361 euclid.aos/1183667291[42] Walker, S. G., Lijoi, A. and Prünster, I. (2007). On rates of convergence for posterior distributions in infinite-dimensional models. Ann. Statist. 35 738–746. 1117.62047 10.1214/009053606000001361 euclid.aos/1183667291
[43] Wasserman, L. (2000). Bayesian model selection and model averaging. J. Math. Psych. 44 92–107. 0946.62032 10.1006/jmps.1999.1278[43] Wasserman, L. (2000). Bayesian model selection and model averaging. J. Math. Psych. 44 92–107. 0946.62032 10.1006/jmps.1999.1278
[44] Wong, W. H. and Shen, X. (1995). Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. Ann. Statist. 23 339–362. 0829.62002 10.1214/aos/1176324524 euclid.aos/1176324524[44] Wong, W. H. and Shen, X. (1995). Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. Ann. Statist. 23 339–362. 0829.62002 10.1214/aos/1176324524 euclid.aos/1176324524
