February 2025 Nearest neighbor empirical processes
François Portier
Author Affiliations +
Bernoulli 31(1): 312-332 (February 2025). DOI: 10.3150/24-BEJ1729

Abstract

In the regression framework, the empirical measure based on the responses resulting from the nearest neighbors, among the covariates, to a given point x is introduced and studied as a central statistical quantity. First, the associated empirical process is shown to satisfy a uniform central limit theorem under a local bracketing entropy condition on the underlying class of functions reflecting the localizing nature of the nearest neighbor algorithm. Second a uniform non-asymptotic bound is established under a well-known condition, often referred to as Vapnik-Chervonenkis, on the uniform entropy numbers. The covariance of the Gaussian limit obtained in the uniform central limit theorem is simply equal to the conditional covariance operator given the covariate value. This suggests the possibility of using standard formulas to estimate the variance by using only the nearest neighbors instead of the full data. This is illustrated on two problems: the estimation of the conditional cumulative distribution function and local linear regression.

Version Information

The current online version of this article, posted on 11 December 2024, supersedes both the original publication version posted on 30 October 2024 and the version appearing in print copies of volume 31, number 1. Corrections have been made to Lemma 3.

Acknowledgments

The author would like to express his gratitude to Johan Segers and Aigerim Zhuman for spotting a mistake in the statement of Lemma 3 in an earlier version of the paper.

Citation

Download Citation

François Portier. "Nearest neighbor empirical processes." Bernoulli 31 (1) 312 - 332, February 2025. https://doi.org/10.3150/24-BEJ1729

Information

Received: 1 March 2023; Published: February 2025
First available in Project Euclid: 30 October 2024

Digital Object Identifier: 10.3150/24-BEJ1729

Keywords: concentration inequality , Empirical process theory , nearest neighbor algorithm , weak convergence

Vol.31 • No. 1 • February 2025
Back to Top