Information inequalities and concentration of measure

Abstract

We derive inequalities of the form $\Delta (P, Q) \leq H(P|R) + H(Q|R)$ which hold for every choice of probability measures P, Q, R, where $H(P|R)$ denotes the relative entropy of $P$ with respect to $R$ and $\Delta (P, Q)$ stands for a coupling type "distance" between $P$ and $Q$. Using the chain rule for relative entropies and then specializing to $Q$ with a given support we recover some of Talagrand's concentration of measure inequalities for product spaces.