Channel Entropy and Primitive Approximation

Abstract

For stationary channels, channel entropy is defined as the supremum of the conditional output entropy over all stationary input sources. If the channel is $\bar{d}$-continuous and conditionally almost block independent, channel entropy gives an upper bound to the output rate of the channel. Furthermore, the channel can be approximated arbitrarily well in the $\bar{d}$-metric by any primitive channel whose noise source entropy exceeds channel entropy and cannot be so approximated if the noise source entropy is less than channel entropy.