The Annals of Applied Probability

Rank inversions in scoring multipart examinations

Simeon M. Berman

Full-text: Open access


Let $(X_i, Y_i), i = 1, \dots, n$, be independent random vectors with a standard bivariate normal distribution and let $s_X$ and $s_Y$ be the sample standard deviations. For arbitrary $p, 0 < p < 1$, define $T_i = pX_i + (1 - p) Y_i$ and $Z_i = pX_i / s_X + (1 - p) Y_i / s_Y, i = 1, \dots, n$. The couple of pairs $(T_i, Z_i)$ and $(T_j, Z_j)$ is said to be discordant if either $T_i < T_j$ and $Z_i > Z_j$ or $T_i > T_j$ and $Z_i < Z_j$. It is shown that the expected number of discordant couples of pairs is asymptotically equal to $n^{3/2}$ times an explicit constant depending on p and the correlation coefficient of $X_i$ and $Y_i$. By an application of the Durbin-Stuart inequality, this implies an asymptotic lower bound on the expected value of the sum of $(\rank(Z_i) - \rank(T_i))^+$. The problem arose in a court challenge to a standard procedure for the scoring of multipart written civil service examinations. Here the sum of the positive rank differences represents a measure of the unfairness of the method of scoring.

Article information

Ann. Appl. Probab., Volume 6, Number 3 (1996), 992-1005.

First available in Project Euclid: 18 October 2002

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F07: Ranking and selection 62H20: Measures of association (correlation, canonical correlation, etc.)

Bivariate normal distribution concordance ranks sample variance


Berman, Simeon M. Rank inversions in scoring multipart examinations. Ann. Appl. Probab. 6 (1996), no. 3, 992--1005. doi:10.1214/aoap/1034968237.

Export citation


  • [1] Cram´er, H. (1946). Mathematical Methods of Statistics. Princeton Univ. Press.
  • [2] Durbin, J. and Stuart, A. (1951). Inversions and rank correlation coefficients. J. Roy. Statist. Soc. Ser. B 13.
  • [3] Evans, M. (1993). Supreme Court of the State of New York, County of New York, IAS Part 36, Index No. 8623/92. Filed September 3, 1993, County Clerk's Office, New York.
  • [4] Kendall, M. G. (1962). Rank Correlation Methods, 3rd ed. Charles Griffin, London.