Translator Disclaimer
December 2020 Optimal rates of entropy estimation over Lipschitz balls
Yanjun Han, Jiantao Jiao, Tsachy Weissman, Yihong Wu
Ann. Statist. 48(6): 3228-3250 (December 2020). DOI: 10.1214/19-AOS1927


We consider the problem of minimax estimation of the entropy of a density over Lipschitz balls. Dropping the usual assumption that the density is bounded away from zero, we obtain the minimax rates $(n\ln n)^{-s/(s+d)}+n^{-1/2}$ for $0<s\leq 2$ for densities supported on $[0,1]^{d}$, where $s$ is the smoothness parameter and $n$ is the number of independent samples. We generalize the results to densities with unbounded support: given an Orlicz functions $\Psi $ of rapid growth (such as the subexponential and sub-Gaussian classes), the minimax rates for densities with bounded $\Psi $-Orlicz norm increase to $(n\ln n)^{-s/(s+d)}(\Psi ^{-1}(n))^{d(1-d/p(s+d))}+n^{-1/2}$, where $p$ is the norm parameter in the Lipschitz ball. We also show that the integral-form plug-in estimators with kernel density estimates fail to achieve the minimax rates, and characterize their worst case performances over the Lipschitz ball.

One of the key steps in analyzing the bias relies on a novel application of the Hardy–Littlewood maximal inequality, which also leads to a new inequality on the Fisher information that may be of independent interest.


Download Citation

Yanjun Han. Jiantao Jiao. Tsachy Weissman. Yihong Wu. "Optimal rates of entropy estimation over Lipschitz balls." Ann. Statist. 48 (6) 3228 - 3250, December 2020.


Received: 1 October 2018; Revised: 1 November 2019; Published: December 2020
First available in Project Euclid: 11 December 2020

Digital Object Identifier: 10.1214/19-AOS1927

Primary: 62G05
Secondary: 62C20

Keywords: Minimax rate , nonparametrics , nonsmooth functional estimation , polynomial approximation

Rights: Copyright © 2020 Institute of Mathematical Statistics


This article is only available to subscribers.
It is not available for individual sale.

Vol.48 • No. 6 • December 2020
Back to Top