Open Access
November 2011 Multiple Testing for Exploratory Research
Jelle J. Goeman, Aldo Solari
Statist. Sci. 26(4): 584-597 (November 2011). DOI: 10.1214/11-STS356

Abstract

Motivated by the practice of exploratory research, we formulate an approach to multiple testing that reverses the conventional roles of the user and the multiple testing procedure. Traditionally, the user chooses the error criterion, and the procedure the resulting rejected set. Instead, we propose to let the user choose the rejected set freely, and to let the multiple testing procedure return a confidence statement on the number of false rejections incurred. In our approach, such confidence statements are simultaneous for all choices of the rejected set, so that post hoc selection of the rejected set does not compromise their validity. The proposed reversal of roles requires nothing more than a review of the familiar closed testing procedure, but with a focus on the non-consonant rejections that this procedure makes. We suggest several shortcuts to avoid the computational problems associated with closed testing.

Citation

Download Citation

Jelle J. Goeman. Aldo Solari. "Multiple Testing for Exploratory Research." Statist. Sci. 26 (4) 584 - 597, November 2011. https://doi.org/10.1214/11-STS356

Information

Published: November 2011
First available in Project Euclid: 28 February 2012

zbMATH: 1331.62369
MathSciNet: MR2951390
Digital Object Identifier: 10.1214/11-STS356

Keywords: Closed testing , confidence set , false discovery proportion

Rights: Copyright © 2011 Institute of Mathematical Statistics

Vol.26 • No. 4 • November 2011
Back to Top