Translator Disclaimer
December 2020 Optimal rates of entropy estimation over Lipschitz balls
Yanjun Han, Jiantao Jiao, Tsachy Weissman, Yihong Wu
Ann. Statist. 48(6): 3228-3250 (December 2020). DOI: 10.1214/19-AOS1927

Abstract

We consider the problem of minimax estimation of the entropy of a density over Lipschitz balls. Dropping the usual assumption that the density is bounded away from zero, we obtain the minimax rates $(n\ln n)^{-s/(s+d)}+n^{-1/2}$ for $0<s\leq 2$ for densities supported on $[0,1]^{d}$, where $s$ is the smoothness parameter and $n$ is the number of independent samples. We generalize the results to densities with unbounded support: given an Orlicz functions $\Psi $ of rapid growth (such as the subexponential and sub-Gaussian classes), the minimax rates for densities with bounded $\Psi $-Orlicz norm increase to $(n\ln n)^{-s/(s+d)}(\Psi ^{-1}(n))^{d(1-d/p(s+d))}+n^{-1/2}$, where $p$ is the norm parameter in the Lipschitz ball. We also show that the integral-form plug-in estimators with kernel density estimates fail to achieve the minimax rates, and characterize their worst case performances over the Lipschitz ball.

One of the key steps in analyzing the bias relies on a novel application of the Hardy–Littlewood maximal inequality, which also leads to a new inequality on the Fisher information that may be of independent interest.

Citation

Download Citation

Yanjun Han. Jiantao Jiao. Tsachy Weissman. Yihong Wu. "Optimal rates of entropy estimation over Lipschitz balls." Ann. Statist. 48 (6) 3228 - 3250, December 2020. https://doi.org/10.1214/19-AOS1927

Information

Received: 1 October 2018; Revised: 1 November 2019; Published: December 2020
First available in Project Euclid: 11 December 2020

Digital Object Identifier: 10.1214/19-AOS1927

Subjects:
Primary: 62G05
Secondary: 62C20

Rights: Copyright © 2020 Institute of Mathematical Statistics

JOURNAL ARTICLE
23 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

SHARE
Vol.48 • No. 6 • December 2020
Back to Top