Open Access
Translator Disclaimer
June 2016 Classification in general finite dimensional spaces with the k-nearest neighbor rule
Sébastien Gadat, Thierry Klein, Clément Marteau
Ann. Statist. 44(3): 982-1009 (June 2016). DOI: 10.1214/15-AOS1395


Given an $n$-sample of random vectors $(X_{i},Y_{i})_{1\leq i\leq n}$ whose joint law is unknown, the long-standing problem of supervised classification aims to optimally predict the label $Y$ of a given new observation $X$. In this context, the $k$-nearest neighbor rule is a popular flexible and intuitive method in non-parametric situations. Even if this algorithm is commonly used in the machine learning and statistics communities, less is known about its prediction ability in general finite dimensional spaces, especially when the support of the density of the observations is $\mathbb{R}^{d}$. This paper is devoted to the study of the statistical properties of the $k$-nearest neighbor rule in various situations. In particular, attention is paid to the marginal law of $X$, as well as the smoothness and margin properties of the regression function $\eta(X)=\mathbb{E}[Y|X]$. We identify two necessary and sufficient conditions to obtain uniform consistency rates of classification and derive sharp estimates in the case of the $k$-nearest neighbor rule. Some numerical experiments are proposed at the end of the paper to help illustrate the discussion.


Download Citation

Sébastien Gadat. Thierry Klein. Clément Marteau. "Classification in general finite dimensional spaces with the k-nearest neighbor rule." Ann. Statist. 44 (3) 982 - 1009, June 2016.


Received: 1 November 2014; Revised: 1 September 2015; Published: June 2016
First available in Project Euclid: 11 April 2016

zbMATH: 1338.62082
MathSciNet: MR3485951
Digital Object Identifier: 10.1214/15-AOS1395

Primary: 62F15 , 62G05
Secondary: 62G20

Keywords: $k$-nearest neighbor , plug-in rules , Supervised classification rates

Rights: Copyright © 2016 Institute of Mathematical Statistics


Vol.44 • No. 3 • June 2016
Back to Top