The Annals of Statistics

Rank verification for exponential families

Kenneth Hung and William Fithian

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

Many statistical experiments involve comparing multiple population groups. For example, a public opinion poll may ask which of several political candidates commands the most support; a social scientific survey may report the most common of several responses to a question; or, a clinical trial may compare binary patient outcomes under several treatment conditions to determine the most effective treatment. Having observed the “winner” (largest observed response) in a noisy experiment, it is natural to ask whether that candidate, survey response or treatment is actually the “best” (stochastically largest response). This article concerns the problem of rank verification—post hoc significance tests of whether the orderings discovered in the data reflect the population ranks. For exponential family models, we show under mild conditions that an unadjusted two-tailed pairwise test comparing the first two-order statistics (i.e., comparing the “winner” to the “runner-up”) is a valid test of whether the winner is truly the best. We extend our analysis to provide equally simple procedures to obtain lower confidence bounds on the gap between the winning population and the others, and to verify ranks beyond the first.

Article information

Source
Ann. Statist., Volume 47, Number 2 (2019), 758-782.

Dates
Received: February 2017
Revised: June 2017
First available in Project Euclid: 11 January 2019

Permanent link to this document
https://projecteuclid.org/euclid.aos/1547197237

Digital Object Identifier
doi:10.1214/17-AOS1634

Subjects
Primary: 62F07: Ranking and selection
Secondary: 62F03: Hypothesis testing

Keywords
Ranking selective inference exponential family multiple comparison sample best

Citation

Hung, Kenneth; Fithian, William. Rank verification for exponential families. Ann. Statist. 47 (2019), no. 2, 758--782. doi:10.1214/17-AOS1634. https://projecteuclid.org/euclid.aos/1547197237


Export citation

References

  • Alikhani, L. (2011). Study: Tween TV today is all about fame. Available at http://thechart.blogs.cnn.com/2011/08/05/study-tweens-aim-for-fame-above-all-else/.
  • Berger, R. L. (1980). Minimax subset selection for the multinomial distribution. J. Statist. Plann. Inference 4 391–402.
  • Berger, R. L. (1982). Multiparameter hypothesis testing and acceptance sampling. Technometrics 24 295–300.
  • Besag, J. and Clifford, P. (1989). Generalized Monte Carlo significance tests. Biometrika 76 633–642.
  • Bofinger, E. (1991). Selecting “demonstrably best” or “demonstrably worst” exponential populations. Aust. N. Z. J. Stat. 33 183–190.
  • Edwards, D. G. and Hsu, J. C. (1983). Multiple comparisons with the best treatment. J. Amer. Statist. Assoc. 78 965–971.
  • Finner, H. and Strassburger, K. (2002). The partitioning principle: A powerful tool in multiple decision theory. Ann. Statist. 30 1194–1213.
  • Fithian, W., Sun, D. L. and Taylor, J. E. (2014). Optimal inference after model selection. Preprint. Available at arXiv:1410.2597.
  • Fithian, W., Taylor, J. E. and Tibshirani, R. J. (2015). Selective sequential model selection. Preprint. Available at arXiv:1512.02565.
  • Gupta, S. S., Huang, D.-Y. and Panchapakesan, S. (1984). On some inequalities and monotonicity results in selection and ranking theory. In Inequalities in Statistics and Probability (Lincoln, Neb., 1982) 211–227. IMS, Hayward, CA.
  • Gupta, S. S. and Liang, T. (1989). Selecting the best binomial population: Parametric empirical Bayes approach. J. Statist. Plann. Inference 23 21–31.
  • Gupta, S. S. and Nagel, K. (1967). On selection and ranking procedures and order statistics from the multinomial distribution. Sankhyā 29 1–34.
  • Gupta, S. S. and Panchapakesan, S. (1971). On multiple decision (subset selection) procedures. Technical report, Purdue Univ., West Lafayette, IN.
  • Gupta, S. S. and Panchapakesan, S. (1985). Subset selection procedures: Review and assessment. Amer. J. Math. Management Sci. 5 235–311.
  • Gupta, S. S. and Wong, W.-Y. (1976). On subset selection procedures for Poisson processes and some applications to the binomial and multinomial problems. Technical report.
  • Gutmann, S. and Maymin, Z. (1987). Is the selected population the best? Ann. Statist. 15 456–461.
  • Hsu, J. C. (1984). Constrained simultaneous confidence intervals for multiple comparisons with the best. Ann. Statist. 12 1136–1144.
  • Hsu, J. (1996). Multiple Comparisons: Theory and Methods. CRC Press, Boca Raton, FL.
  • Kannan, N. and Panchapakesan, S. (2009). Does the selected normal population have the smallest variance? Amer. J. Math. Management Sci. 29 109–123.
  • Marshall, A. W., Olkin, I. and Arnold, B. C. (2011). Inequalities: Theory of Majorization and Its Applications, 2nd ed. Springer, New York.
  • Maymin, Z. and Gutmann, S. (1992). Testing retrospective hypotheses. Canad. J. Statist. 20 335–345.
  • Nettleton, D. (2009). Testing for the supremacy of a multinomial cell probability. J. Amer. Statist. Assoc. 104 1052–1059.
  • Ng, H. K. T. and Panchapakesan, S. (2007). Is the selected multinomial cell the best?. Sequential Anal. 26 415–423.
  • Quinnipiac University Poll Institute (2016). First-timers put Trump ahead in Iowa GOP caucus, Quinnipiac University poll finds; Sanders needs first-timers to tie Clinton in Dem caucus. Available at https://poll.qu.edu/images/polling/ia/ia02012016_Ifsmb28.pdf/.
  • Stefansson, G., Kim, W.-C. and Hsu, J. C. (1988). On confidence sets in multiple comparisons. In Statistical Decision Theory and Related Topics IV 2 89–104. Springer, New York.
  • Uhls, Y. T. and Greenfield, P. M. (2012). The value of fame: Preadolescent perceptions of popular media and their relationship to future aspirations. Dev. Psychol. 48 315–326.