Open Access
Translator Disclaimer
October 2013 The asymptotics of ranking algorithms
John C. Duchi, Lester Mackey, Michael I. Jordan
Ann. Statist. 41(5): 2292-2323 (October 2013). DOI: 10.1214/13-AOS1142

Abstract

We consider the predictive problem of supervised ranking, where the task is to rank sets of candidate items returned in response to queries. Although there exist statistical procedures that come with guarantees of consistency in this setting, these procedures require that individuals provide a complete ranking of all items, which is rarely feasible in practice. Instead, individuals routinely provide partial preference information, such as pairwise comparisons of items, and more practical approaches to ranking have aimed at modeling this partial preference data directly. As we show, however, such an approach raises serious theoretical challenges. Indeed, we demonstrate that many commonly used surrogate losses for pairwise comparison data do not yield consistency; surprisingly, we show inconsistency even in low-noise settings. With these negative results as motivation, we present a new approach to supervised ranking based on aggregation of partial preferences, and we develop $U$-statistic-based empirical risk minimization procedures. We present an asymptotic analysis of these new procedures, showing that they yield consistency results that parallel those available for classification. We complement our theoretical results with an experiment studying the new procedures in a large-scale web-ranking task.

Citation

Download Citation

John C. Duchi. Lester Mackey. Michael I. Jordan. "The asymptotics of ranking algorithms." Ann. Statist. 41 (5) 2292 - 2323, October 2013. https://doi.org/10.1214/13-AOS1142

Information

Published: October 2013
First available in Project Euclid: 5 November 2013

zbMATH: 1281.62058
MathSciNet: MR3127867
Digital Object Identifier: 10.1214/13-AOS1142

Subjects:
Primary: 62C99 , 62F07 , 62F12 , 68Q32

Keywords: $U$-statistics , asymptotics , consistency , Fisher consistency , rank aggregation , ranking

Rights: Copyright © 2013 Institute of Mathematical Statistics

JOURNAL ARTICLE
32 PAGES


SHARE
Vol.41 • No. 5 • October 2013
Back to Top