Electronic Journal of Statistics

A studentized permutation test for the nonparametric Behrens-Fisher problem in paired data

Frank Konietschke and Markus Pauly

Full-text: Open access


We consider nonparametric ranking methods for matched pairs, whose distributions can have different shapes even under the null hypothesis of no treatment effect. Although the data may not be exchangeable under the null, we investigate a permutation approach as a valid procedure for finite sample sizes. In particular, we derive the limit of the studentized permutation distribution under alternatives, which can be used for the construction of $(1-\alpha)$-confidence intervals. Simulation studies show that the new approach is more accurate than its competitors. The procedures are illustrated using a real data set.

Article information

Electron. J. Statist., Volume 6 (2012), 1358-1372.

First available in Project Euclid: 26 July 2012

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Confidence intervals heteroscedasticity matched pairs nonparametric Behrens-Fisher problem rank statistics resampling


Konietschke, Frank; Pauly, Markus. A studentized permutation test for the nonparametric Behrens-Fisher problem in paired data. Electron. J. Statist. 6 (2012), 1358--1372. doi:10.1214/12-EJS714. https://projecteuclid.org/euclid.ejs/1343310300

Export citation


  • Akritas, M. G., Arnold, S. F. and Brunner, E. (1997). Nonparametric hypotheses and rank statistics for unbalanced factorial designs., Journal of the American Statistical Association 92 258–265.
  • Akritas, S. F. and Brunner, E. (1997). A unified approach to rank tests for mixed models., Journal of Statistical Planning and Inference 61 249–277.
  • Brunner, E., Domhof, S. and Langer, F. (2002)., Nonparametric Analysis of Longitudinal Data in Factorial Experiments. Wiley, New York.
  • Brunner, E. and Munzel, U. (2000). The Nonparametric Behrens-Fisher Problem: Asymptotic Theory and a Small-Sample Approximation., Biometrical Journal 1 17–21.
  • Brunner, E., Puri, M. L. and Sun, S. (1995). Nonparametric methods for stratified two-sample designs with application to multi clinic trials., Journal of the American Statistical Association 90 1004–1014.
  • Brunner, E. and Puri, M. L. (1996). Nonparametric methods in design and analysis of experiments., Handbook of Statistics 13 631–703.
  • ICH, (1998)., Statistical Principles for Clinical Trials ICH, Guideline.
  • Janssen, A. (1997). Studentized permutation test for non-i.i.d. hypotheses and the generalized Behrens – Fisher problem., Statist. Probab. Lett. 36 9–21.
  • Janssen, A. (1999a). Nonparametric symmetry tests for statistical functionals., Math. Meth. Stat. 8 320–343.
  • Janssen, A. (1999b). Testing nonparametric statistical functionals with application to rank tests., J. Stat. Plan. Inference 81 71–93.
  • Janssen, A. (2005). Resampling Student’s t-type statistics., Ann. Inst. Statist. Math. 57 507–529.
  • Janssen, A. and Pauls, T. (2003). How do bootstrap and permutation tests work?, The Annals of Statistics 3 768–806.
  • Konietschke, F., Bathke, A. C., Hothorn, L. A. and Brunner, E. (2010). Testing and estimation of purely nonparametric effects in repeated measures designs., Computational Statistics and Data Analysis 54 1895–1905.
  • Krone, B., Pohl, D., Rostasy, K., Kahler, E., Brunner, E., Oeffner, F., Grange, J. M., Gaertner, J. and Hanefeld, F. (2008). Common infectious agents in multiple sclerosis: a case-control study in children., Multiple Sclerosis Journal 14 136–139.
  • Munzel, U. (1999a). Linear rank score statistics when ties are present., Statistics and Probability Letters 41 389–395.
  • Munzel, U. (1999b). Nonparametric methods for paired samples., Statistica Neerlandica 53 277–286.
  • Munzel, U. and Brunner, E. (2002). An Exact Paired Rank Test., Biometrical Journal 44 584–593.
  • Neubert, K. and Brunner, E. (2007). A studentized permutation test for the non-parametric Behrens – Fisher problem., Computational Statistics and Data Analysis 51 5192–5204.
  • Obenauer, S., Luftner-Nagel, S., von Heyden, D., Munzel, U., Baum, F. and Grabbe, E. (2002). Screen film vs full-field digital mam-mography: image quality, detectability and characterization of lesions., European Radiology 12 1697–1702.
  • Orban, J. and Wolfe, D. A. (1982). A class of distribution-free two-sample tests based on placements., Journal of the American Statistical Association 77 666–672.
  • R Development Core Team, (2010)., R: A Language and Environment for Statistical Computing R Foundation for Statistical Computing, Vienna, Austria ISBN 3-900051-07-0.
  • Ruymgaart, F. H. (1980). A unified approach to the asymptotic distribution theory of certain midrank statistics., Statistique non Parametrique Asymptotique, 118, J.P. Raoult. Lecture Notes on Mathematics 821.
  • Ryu, E. (2009). Simultaneous confidence intervals using ordinal effect measures for ordered categorical outcomes., Statistics In Medicine 28 3179–3188.
  • Shorack, G. S. and Wellner, J. A. (1986)., Empirical Processes with Applications to Statistics. Wiley, New York.