Rank disequilibrium in multiple-criteria evaluation schemes

Abstract

In multiple-criteria evaluation schemes, rank disequilibrium occurs when an evaluee is rated higher than other evaluees on some criteria and lower than other evaluees on other criteria. In this article, we investigate rank disequilibrium as it relates to the problem of aggregating scores on individual criteria into an overall evaluation. We adopt an axiomatic approach, defining the notion of a rank aggregation function and proposing a set of desirable properties — namely, independence, monotonicity, inclusivity, consistency, and equity — that rank aggregation functions may or may not satisfy. We prove that when there are more than three possible scores on each criterion, it is impossible to define a rank aggregation function that satisfies all of these properties. We then investigate potential resolutions to the problems posed by rank disequilibrium.