Statistical Science

Permutation Methods: A Basis for Exact Inference

Michael D. Ernst

Full-text: Open access


The use of permutation methods for exact inference dates back to Fisher in 1935. Since then, the practicality of such methods has increased steadily with computing power. They can now easily be employed in many situations without concern for computing difficulties. We discuss the reasoning behind these methods and describe situations when they are exact and distribution-free. We illustrate their use in several examples.

Article information

Statist. Sci., Volume 19, Number 4 (2004), 676-685.

First available in Project Euclid: 18 April 2005

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Distribution-free Monte Carlo nonparametric permutation tests randomization tests


Ernst, Michael D. Permutation Methods: A Basis for Exact Inference. Statist. Sci. 19 (2004), no. 4, 676--685. doi:10.1214/088342304000000396.

Export citation


  • Bradley, J. V. (1968). Distribution-Free Statistical Tests. Prentice-Hall, Englewood Cliffs, NJ.
  • Cytel Software Corporation (2003). StatXact 6. Cytel Software Corporation, Cambridge, MA.
  • Dwass, M. (1957). Modified randomization tests for nonparametric hypotheses. Ann. Math. Statist. 28 181--187.
  • Edgington, E. S. (1995). Randomization Tests, 3rd ed. Dekker, New York.
  • Ernst, M. D. and Schucany, W. R. (1999). A class of permutation tests of bivariate interchangeability. J. Amer. Statist. Assoc. 94 273--284.
  • Fisher, R. A. (1935). The Design of Experiments. Oliver and Boyd, Edinburgh.
  • Fisher, R. A. (1936). ``The coefficient of racial likeness'' and the future of craniometry. J. Royal Anthropological Institute of Great Britain and Ireland 66 57--63.
  • Garthwaite, P. H. (1996). Confidence intervals from randomization tests. Biometrics 52 1387--1393.
  • Grogan, W. L., Jr. and Wirth, W. W. (1981). A new American genus of predaceous midges related to Palpomyia and Bezzia (Diptera: Ceratopogonidae). Proc. Biological Society of Washington 94 1279--1305.
  • Higgins, J. J. (2004). An Introduction to Modern Nonparametric Statistics. Brooks/Cole, Pacific Grove, CA.
  • Ihaka, R. and Gentleman, R. (1996). R: A language for data analysis and graphics. J. Comput. Graph. Statist. 5 299--314.
  • Jöckel, K.-H. (1986). Finite sample properties and asymptotic efficiency of Monte Carlo tests. Ann. Statist. 14 336--347.
  • Kennedy, P. E. and Cade, B. S. (1996). Randomization tests for multiple regression. Comm. Statist. Simulation Comput. 25 923--936.
  • Lehmann, E. L. (1975). Nonparametrics: Statistical Methods Based on Ranks. Holden--Day, San Francisco.
  • Manly, B. F. J. (1997). Randomization, Bootstrap and Monte Carlo Methods in Biology, 2nd ed. Chapman and Hall, London.
  • Pitman, E. J. G. (1937a). Significance tests which may be applied to samples from any populations. J. Roy. Statist. Soc. Suppl. 4 119--130.
  • Pitman, E. J. G. (1937b). Significance tests which may be applied to samples from any populations. II. The correlation coefficient test. J. Roy. Statist. Soc. Suppl. 4 225--232.
  • Pitman, E. J. G. (1938). Significance tests which may be applied to samples from any populations. III. The analysis of variance test. Biometrika 29 322--335.