A lower bound on the relative entropy with respect to a symmetric probability

Abstract

Let $\rho$ and $\mu$ be two probability measures on $\mathbb{R}$ which are not the Dirac mass at $0$. We denote by $H(\mu|\rho)$ the relative entropy of $\mu$ with respect to $\rho$. We prove that, if $\rho$ is symmetric and $\mu$ has a finite first moment, then \[ H(\mu|\rho)\geq \frac{\displaystyle{\left(\int_{\mathbb{R}}z\,d\mu(z)\right)^2}}{\displaystyle{2\int_{\mathbb{R}}z^2\,d\mu(z)}}\,,\] with equality if and only if $\mu=\rho$. We give an applicaion to the Curie-Weiss model of self-organized criticality.