Kleeman has recently demonstrated that the relative entropyprovides a significant measure of the information content of aprediction ensemble compared with the climate record in severalsimplified climate models. Here several additional aspects ofutilizing the relative entropy for predictability theory aredeveloped with full mathematical rigor in a systematic fashionwhich the authors believe will be very useful in practicalproblems with many degrees of freedom in atmosphere/oceanand biological science. The results developed here include ageneralized signal-dispersion decomposition, rigorous explicitlower bound estimators for information content, and rigorouslower bound estimates on relative entropy for many variables,N, through N, one-dimensional relative entropiesand N, two-dimensional mutual information functions.These last results provide a practical context for rapidevaluation of the predictive information content in a largenumber of variables.
"A Mathematical Framework for Quantifying Predictability Through Relative Entropy." Methods Appl. Anal. 9 (3) 425 - 444, September 2002.