Open Access
Translator Disclaimer
October 2008 A class of Rényi information estimators for multidimensional densities
Nikolai Leonenko, Luc Pronzato, Vippal Savani
Ann. Statist. 36(5): 2153-2182 (October 2008). DOI: 10.1214/07-AOS539

Abstract

A class of estimators of the Rényi and Tsallis entropies of an unknown distribution f in ℝm is presented. These estimators are based on the kth nearest-neighbor distances computed from a sample of N i.i.d. vectors with distribution f. We show that entropies of any order q, including Shannon’s entropy, can be estimated consistently with minimal assumptions on f. Moreover, we show that it is straightforward to extend the nearest-neighbor method to estimate the statistical distance between two distributions using one i.i.d. sample from each.

Citation

Download Citation

Nikolai Leonenko. Luc Pronzato. Vippal Savani. "A class of Rényi information estimators for multidimensional densities." Ann. Statist. 36 (5) 2153 - 2182, October 2008. https://doi.org/10.1214/07-AOS539

Information

Published: October 2008
First available in Project Euclid: 13 October 2008

zbMATH: 1205.94053
MathSciNet: MR2458183
Digital Object Identifier: 10.1214/07-AOS539

Subjects:
Primary: 62G20 , 94A15

Keywords: Entropy estimation , estimation of divergence , estimation of statistical distance , Havrda–Charvát entropy , nearest-neighbor distances , Rényi entropy , Tsallis entropy

Rights: Copyright © 2008 Institute of Mathematical Statistics

JOURNAL ARTICLE
30 PAGES


SHARE
Vol.36 • No. 5 • October 2008
Back to Top