The Annals of Statistics

Approximation of Density Functions by Sequences of Exponential Families

Andrew R. Barron and Chyong-Hwa Sheu

Full-text: Open access

Abstract

Probability density functions are estimated by the method of maximum likelihood in sequences of regular exponential families. This method is also familiar as entropy maximization subject to empirical constraints. The approximating families of log-densities that we consider are polynomials, splines and trigonometric series. Bounds on the relative entropy (Kullback-Leibler distance) between the true density and the estimator are obtained and rates of convergence are established for log-density functions assumed to have square integrable derivatives.

Article information

Source
Ann. Statist., Volume 19, Number 3 (1991), 1347-1369.

Dates
First available in Project Euclid: 12 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aos/1176348252

Digital Object Identifier
doi:10.1214/aos/1176348252

Mathematical Reviews number (MathSciNet)
MR1126328

Zentralblatt MATH identifier
0739.62027

JSTOR
links.jstor.org

Subjects
Primary: 62G05: Estimation
Secondary: 41A17: Inequalities in approximation (Bernstein, Jackson, Nikol s kii-type inequalities) 62B10: Information-theoretic topics [See also 94A17] 62F12: Asymptotic properties of estimators

Keywords
Log-density estimation exponential families minimum relative entropy estimation Kullback-Leibler number $L_2$ approximation

Citation

Barron, Andrew R.; Sheu, Chyong-Hwa. Approximation of Density Functions by Sequences of Exponential Families. Ann. Statist. 19 (1991), no. 3, 1347--1369. doi:10.1214/aos/1176348252. https://projecteuclid.org/euclid.aos/1176348252


Export citation

Corrections

  • See Correction: Andrew R. Barron, Chyong-Hwa Sheu. Correction: Approximation of Density Functions by Sequences of Exponential Families. Ann. Statist., Volume 19, Number 4 (1991), 2284--2284.