Relative log-concavity and a pair of triangle inequalities

Abstract

The relative log-concavity ordering ≤_lc between probability mass functions (pmf’s) on non-negative integers is studied. Given three pmf’s f, g, h that satisfy f ≤_lc g ≤_lc h, we present a pair of (reverse) triangle inequalities: if ∑_iif_i=∑_iig_i<∞, then

D(f|h)≥D(f|g)+D(g|h)

and if ∑_iig_i=∑_iih_i<∞, then

D(h|f)≥D(h|g)+D(g|f),

where D(⋅|⋅) denotes the Kullback–Leibler divergence. These inequalities, interesting in themselves, are also applied to several problems, including maximum entropy characterizations of Poisson and binomial distributions and the best binomial approximation in relative entropy. We also present parallel results for continuous distributions and discuss the behavior of ≤_lc under convolution.