Open Access
2019 Non-marginal decisions: A novel Bayesian multiple testing procedure
Noirrit Kiran Chandra, Sourabh Bhattacharya
Electron. J. Statist. 13(1): 489-535 (2019). DOI: 10.1214/19-EJS1535
Abstract

In this paper, we consider the problem of multiple testing where the hypotheses are dependent. In most of the existing literature, either Bayesian or non-Bayesian, the decision rules mainly focus on the validity of the test procedure rather than actually utilizing the dependency to increase efficiency. Moreover, the decisions regarding different hypotheses are marginal in the sense that they do not depend upon each other directly. However, in realistic situations, the hypotheses are usually dependent, and hence it is desirable that the decisions regarding the dependent hypotheses are taken jointly.

In this article, we develop a novel Bayesian multiple testing procedure that coherently takes this requirement into consideration. Our method, which is based on new notions of error and non-error terms, substantially enhances efficiency by judicious exploitation of the dependence structure among the hypotheses. We show that our method minimizes the posterior expected loss associated with an additive “0-1” loss function; we also prove theoretical results on the relevant error probabilities, establishing the coherence and usefulness of our method. The optimal decision configuration is not available in closed form and we propose an efficient simulated annealing algorithm for the purpose of optimization, which is also generically applicable to binary optimization problems.

Extensive simulation studies indicate that in dependent situations, our method performs significantly better than some existing popular conventional multiple testing methods, in terms of accuracy and power control. Moreover, application of our ideas to a real, spatial data set associated with radionuclide concentration in Rongelap islands yielded insightful results.

References

1.

Abramovich, F. and Angelini, C. (2006). Bayesian Maximum a posteriori Multiple Testing Procedure., Sankhyā: The Indian Journal of Statistics (2003-2007), 68(3), 436–460. 1193.62031Abramovich, F. and Angelini, C. (2006). Bayesian Maximum a posteriori Multiple Testing Procedure., Sankhyā: The Indian Journal of Statistics (2003-2007), 68(3), 436–460. 1193.62031

2.

Andrieu, C., Breyer, L. A., and Doucet, A. (2001). Convergence of simulated annealing using Foster-Lyapunov criteria., Journal of Applied Probability, 38(4), 975–994. 0999.60066 10.1239/jap/1011994186 euclid.jap/1011994186Andrieu, C., Breyer, L. A., and Doucet, A. (2001). Convergence of simulated annealing using Foster-Lyapunov criteria., Journal of Applied Probability, 38(4), 975–994. 0999.60066 10.1239/jap/1011994186 euclid.jap/1011994186

3.

Benjamini, Y. and Heller, R. (2007). False Discovery Rates for Spatial Signals., Journal of the American Statistical Association, 102(480), 1272–1281. 1332.94019 10.1198/016214507000000941Benjamini, Y. and Heller, R. (2007). False Discovery Rates for Spatial Signals., Journal of the American Statistical Association, 102(480), 1272–1281. 1332.94019 10.1198/016214507000000941

4.

Benjamini, Y. and Hochberg, Y. (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing., Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 289–300. 0809.62014 10.1111/j.2517-6161.1995.tb02031.xBenjamini, Y. and Hochberg, Y. (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing., Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 289–300. 0809.62014 10.1111/j.2517-6161.1995.tb02031.x

5.

Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency., Ann. Statist., 29(4), 1165–1188. 1041.62061 10.1214/aos/1013699998 euclid.aos/1013699998Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency., Ann. Statist., 29(4), 1165–1188. 1041.62061 10.1214/aos/1013699998 euclid.aos/1013699998

6.

Berry, D. A. and Hochberg, Y. (1999). Bayesian perspectives on multiple comparisons., Journal of Statistical Planning and Inference, 82(1), 215–227. 1063.62527 10.1016/S0378-3758(99)00044-0Berry, D. A. and Hochberg, Y. (1999). Bayesian perspectives on multiple comparisons., Journal of Statistical Planning and Inference, 82(1), 215–227. 1063.62527 10.1016/S0378-3758(99)00044-0

7.

Chandra, N. K. and Bhattacharya, S. (2018). Asymptotic theory of a bayesian non-marginal multiple testing procedure under possible model misspecification., arXiv preprint arXiv:1611.01369.Chandra, N. K. and Bhattacharya, S. (2018). Asymptotic theory of a bayesian non-marginal multiple testing procedure under possible model misspecification., arXiv preprint arXiv:1611.01369.

8.

Chandra, N. K., Singh, R., and Bhattacharya, S. (2018). A Novel Bayesian Multiple Testing Approach to Deregulated miRNA Discovery Harnessing Positional Clustering., Biometrics.Chandra, N. K., Singh, R., and Bhattacharya, S. (2018). A Novel Bayesian Multiple Testing Approach to Deregulated miRNA Discovery Harnessing Positional Clustering., Biometrics.

9.

Dey, K. K. and Bhattacharya, S. (2017). A Brief Tutorial on Transformation based Markov Chain Monte Carlo and Optimal Scaling of the Additive Transformation., Braz. J. Probab. Stat., 31(3), 569–617. 1378.60100 10.1214/16-BJPS325 euclid.bjps/1503388830Dey, K. K. and Bhattacharya, S. (2017). A Brief Tutorial on Transformation based Markov Chain Monte Carlo and Optimal Scaling of the Additive Transformation., Braz. J. Probab. Stat., 31(3), 569–617. 1378.60100 10.1214/16-BJPS325 euclid.bjps/1503388830

10.

Diggle, P. J., Tawn, J., and Moyeed, R. (1998). Model-based geostatistics., Journal of the Royal Statistical Society: Series C (Applied Statistics), 47(3), 299–350. MR1626544 0904.62119 10.1111/1467-9876.00113Diggle, P. J., Tawn, J., and Moyeed, R. (1998). Model-based geostatistics., Journal of the Royal Statistical Society: Series C (Applied Statistics), 47(3), 299–350. MR1626544 0904.62119 10.1111/1467-9876.00113

11.

Dudoit, S., Shaffer, J. P., and Boldrick, J. C. (2003). Multiple hypothesis testing in microarray experiments., Statist. Sci., 18(1), 71–103. 1048.62099 10.1214/ss/1056397487 euclid.ss/1056397487Dudoit, S., Shaffer, J. P., and Boldrick, J. C. (2003). Multiple hypothesis testing in microarray experiments., Statist. Sci., 18(1), 71–103. 1048.62099 10.1214/ss/1056397487 euclid.ss/1056397487

12.

Dutta, S. and Bhattacharya, S. (2014). Markov Chain Monte Carlo based on deterministic transformations., Statistical Methodology, 16, 100–116. 07035551 10.1016/j.stamet.2013.08.006Dutta, S. and Bhattacharya, S. (2014). Markov Chain Monte Carlo based on deterministic transformations., Statistical Methodology, 16, 100–116. 07035551 10.1016/j.stamet.2013.08.006

13.

Efron, B. (2007). Correlation and Large-Scale Simultaneous Significance Testing., Journal of the American Statistical Association, 102(477), 93–103. 1284.62340 10.1198/016214506000001211Efron, B. (2007). Correlation and Large-Scale Simultaneous Significance Testing., Journal of the American Statistical Association, 102(477), 93–103. 1284.62340 10.1198/016214506000001211

14.

Fan, J., Han, X., and Gu, W. (2012). Estimating False Discovery Proportion Under Arbitrary Covariance Dependence., Journal of the American Statistical Association, 107(499), 1019–1035. PMID: 24729644. 1395.62219 10.1080/01621459.2012.720478Fan, J., Han, X., and Gu, W. (2012). Estimating False Discovery Proportion Under Arbitrary Covariance Dependence., Journal of the American Statistical Association, 107(499), 1019–1035. PMID: 24729644. 1395.62219 10.1080/01621459.2012.720478

15.

Finner, H. and Roters, M. (2002). Multiple hypotheses testing and expected number of type I. errors., Ann. Statist., 30(1), 220–238. 1012.62020 10.1214/aos/1015362191 euclid.aos/1015362191Finner, H. and Roters, M. (2002). Multiple hypotheses testing and expected number of type I. errors., Ann. Statist., 30(1), 220–238. 1012.62020 10.1214/aos/1015362191 euclid.aos/1015362191

16.

Finner, H., Dickhaus, T., and Roters, M. (2007). Dependency and false discovery rate: Asymptotics., Ann. Statist., 35(4), 1432–1455. 1125.62076 10.1214/009053607000000046 euclid.aos/1188405617Finner, H., Dickhaus, T., and Roters, M. (2007). Dependency and false discovery rate: Asymptotics., Ann. Statist., 35(4), 1432–1455. 1125.62076 10.1214/009053607000000046 euclid.aos/1188405617

17.

Genovese, C. R., Roeder, K., and Wasserman, L. (2006). False Discovery Control with p-Value Weighting., Biometrika, 93(3), 509–524. 1108.62070 10.1093/biomet/93.3.509Genovese, C. R., Roeder, K., and Wasserman, L. (2006). False Discovery Control with p-Value Weighting., Biometrika, 93(3), 509–524. 1108.62070 10.1093/biomet/93.3.509

18.

Guindani, M., Müller, P., and Zhang, S. (2009). A bayesian discovery procedure., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(5), 905–925. 1411.62224 10.1111/j.1467-9868.2009.00714.xGuindani, M., Müller, P., and Zhang, S. (2009). A bayesian discovery procedure., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(5), 905–925. 1411.62224 10.1111/j.1467-9868.2009.00714.x

19.

Heller, R., Stanley, D., Yekutieli, D., Rubin, N., and Benjamini, Y. (2006). Cluster-based analysis of FMRI data., NeuroImage, 33(2), 599–608.Heller, R., Stanley, D., Yekutieli, D., Rubin, N., and Benjamini, Y. (2006). Cluster-based analysis of FMRI data., NeuroImage, 33(2), 599–608.

20.

Jaccard, P. (1901). Étude Comparative de la Distribution Florale dans une Portion des Alpes et des Jura., Bulletin de la Société Vaudoise des Sciences Naturelles, 37, 547–579.Jaccard, P. (1901). Étude Comparative de la Distribution Florale dans une Portion des Alpes et des Jura., Bulletin de la Société Vaudoise des Sciences Naturelles, 37, 547–579.

21.

Jaccard, P. (1908). Nouvelles Recherches sur la Distribution Florale., Bulletin de la Société Vaudoise des Sciences Naturelles, 44, 223–270.Jaccard, P. (1908). Nouvelles Recherches sur la Distribution Florale., Bulletin de la Société Vaudoise des Sciences Naturelles, 44, 223–270.

22.

Jaccard, P. (1912). The Distribution of the Flora in the Alpine Zone., New Phytologist, 11, 37–50.Jaccard, P. (1912). The Distribution of the Flora in the Alpine Zone., New Phytologist, 11, 37–50.

23.

Müller, P., Parmigiani, G., Robert, C., and Rousseau, J. (2004). Optimal sample size for multiple testing: the case of gene expression microarrays., Journal of the American Statistical Association, 99(468), 990–1001. 1055.62127 10.1198/016214504000001646Müller, P., Parmigiani, G., Robert, C., and Rousseau, J. (2004). Optimal sample size for multiple testing: the case of gene expression microarrays., Journal of the American Statistical Association, 99(468), 990–1001. 1055.62127 10.1198/016214504000001646

24.

Qiu, X., Lev, K., and Andrei, Y. (2005). Correlation between gene expression levels and limitations of the empirical bayes methodology for finding differentially expressed genes., Statistical Applications in Genetics and Molecular Biology, 4(1), 1–32.Qiu, X., Lev, K., and Andrei, Y. (2005). Correlation between gene expression levels and limitations of the empirical bayes methodology for finding differentially expressed genes., Statistical Applications in Genetics and Molecular Biology, 4(1), 1–32.

25.

Robert, C. and Casella, G. (2013)., Monte Carlo Statistical Methods. Springer Science & Business Media. 1096.62003Robert, C. and Casella, G. (2013)., Monte Carlo Statistical Methods. Springer Science & Business Media. 1096.62003

26.

Sarkar, S. K., Zhou, T., and Ghosh, D. (2008). A general decision theoretic formulation of procedures controlling FDR and FNR from a Bayesian perspective., Statistica Sinica, 18(3), 925–945. 1149.62003Sarkar, S. K., Zhou, T., and Ghosh, D. (2008). A general decision theoretic formulation of procedures controlling FDR and FNR from a Bayesian perspective., Statistica Sinica, 18(3), 925–945. 1149.62003

27.

Schwartzman, A. and Lin, X. (2011). The effect of correlation in false discovery rate estimation., Biometrika, 98(1), 199–214. 1215.62071 10.1093/biomet/asq075Schwartzman, A. and Lin, X. (2011). The effect of correlation in false discovery rate estimation., Biometrika, 98(1), 199–214. 1215.62071 10.1093/biomet/asq075

28.

Scott, J. G. and Berger, J. O. (2006). An exploration of aspects of Bayesian multiple testing., Journal of Statistical Planning and Inference, 136(7), 2144 – 2162. 1087.62039 10.1016/j.jspi.2005.08.031Scott, J. G. and Berger, J. O. (2006). An exploration of aspects of Bayesian multiple testing., Journal of Statistical Planning and Inference, 136(7), 2144 – 2162. 1087.62039 10.1016/j.jspi.2005.08.031

29.

Scott, J. G. and Berger, J. O. (2010). Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem., Ann. Statist., 38(5), 2587–2619. 1200.62020 10.1214/10-AOS792 euclid.aos/1278861454Scott, J. G. and Berger, J. O. (2010). Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem., Ann. Statist., 38(5), 2587–2619. 1200.62020 10.1214/10-AOS792 euclid.aos/1278861454

30.

Shalizi, C. R. (2009). Dynamics of Bayesian Updating with Dependent Data and Misspecified Models., Electron. J. Statist., 3, 1039–1074. 1326.62017 10.1214/09-EJS485Shalizi, C. R. (2009). Dynamics of Bayesian Updating with Dependent Data and Misspecified Models., Electron. J. Statist., 3, 1039–1074. 1326.62017 10.1214/09-EJS485

31.

Storey, J. D. (2002). A direct approach to false discovery rates., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(3), 479–498. MR1924302 1090.62073 10.1111/1467-9868.00346Storey, J. D. (2002). A direct approach to false discovery rates., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(3), 479–498. MR1924302 1090.62073 10.1111/1467-9868.00346

32.

Storey, J. D. (2003). The positive false discovery rate: a Bayesian interpretation and the q-value., Ann. Statist., 31(6), 2013–2035. 1042.62026 10.1214/aos/1074290335 euclid.aos/1074290335Storey, J. D. (2003). The positive false discovery rate: a Bayesian interpretation and the q-value., Ann. Statist., 31(6), 2013–2035. 1042.62026 10.1214/aos/1074290335 euclid.aos/1074290335

33.

Sun, W. and Cai, T. T. (2009). Large-scale multiple testing under dependence., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(2), 393–424. MR2649603 1248.62005 10.1111/j.1467-9868.2008.00694.xSun, W. and Cai, T. T. (2009). Large-scale multiple testing under dependence., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(2), 393–424. MR2649603 1248.62005 10.1111/j.1467-9868.2008.00694.x

34.

Sun, W., Reich, B. J., Tony Cai, T., Guindani, M., and Schwartzman, A. (2015). False discovery control in large-scale spatial multiple testing., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 77(1), 59–83. 07066221 10.1111/rssb.12064Sun, W., Reich, B. J., Tony Cai, T., Guindani, M., and Schwartzman, A. (2015). False discovery control in large-scale spatial multiple testing., Journal of the Royal Statistical Society: Series B (Statistical Methodology), 77(1), 59–83. 07066221 10.1111/rssb.12064

35.

Xie, J., Cai, T. T., Maris, J., and Li, H. (2011). Optimal false discovery rate control for dependent data., Statistics and its interface, 4(4), 417. 1245.62091 10.4310/SII.2011.v4.n4.a1Xie, J., Cai, T. T., Maris, J., and Li, H. (2011). Optimal false discovery rate control for dependent data., Statistics and its interface, 4(4), 417. 1245.62091 10.4310/SII.2011.v4.n4.a1

36.

Zhang, C., Fan, J., and Yu, T. (2011). Multiple testing via FDR$_l$ for large scale imaging data., Ann. Statist., 39(1), 613–642. 1209.62166 10.1214/10-AOS848 euclid.aos/1297779858Zhang, C., Fan, J., and Yu, T. (2011). Multiple testing via FDR$_l$ for large scale imaging data., Ann. Statist., 39(1), 613–642. 1209.62166 10.1214/10-AOS848 euclid.aos/1297779858
Noirrit Kiran Chandra and Sourabh Bhattacharya "Non-marginal decisions: A novel Bayesian multiple testing procedure," Electronic Journal of Statistics 13(1), 489-535, (2019). https://doi.org/10.1214/19-EJS1535
Received: 1 June 2018; Published: 2019
Vol.13 • No. 1 • 2019
Back to Top