The Annals of Applied Probability

Some relations between mutual information and estimation error in Wiener space

Eddy Mayer-Wolf and Moshe Zakai

Full-text: Open access


The model considered is that of “signal plus white noise.” Known connections between the noncausal filtering error and mutual information are combined with new ones involving the causal estimation error, in a general abstract setup. The results are shown to be invariant under a wide class of causality patterns; they are applied to the derivation of the causal estimation error of a Gaussian nonstationary filtering problem and to a multidimensional extension of the Yovits–Jackson formula.

Article information

Ann. Appl. Probab., Volume 17, Number 3 (2007), 1102-1116.

First available in Project Euclid: 22 May 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60G35: Signal detection and filtering [See also 62M20, 93E10, 93E11, 94Axx] 94A17: Measures of information, entropy
Secondary: 28C20: Set functions and measures and integrals in infinite-dimensional spaces (Wiener measure, Gaussian measure, etc.) [See also 46G12, 58C35, 58D20, 60B11] 60H35: Computational methods for stochastic equations [See also 65C30]

Estimation errors mutual information abstract Wiener space resolution of identity causality


Mayer-Wolf, Eddy; Zakai, Moshe. Some relations between mutual information and estimation error in Wiener space. Ann. Appl. Probab. 17 (2007), no. 3, 1102--1116. doi:10.1214/105051607000000131.

Export citation


  • Duncan, T. E. (1970). On the calculation of mutual information. SIAM J. Appl. Math. 19 215–220.
  • Grenander, U. and Szegö, G. (1958). Toeplitz Forms and Their Applications. Univ. California Press, Berkeley.
  • Guo, D., Shamai, S. and Verdú, S. (2005). Mutual information and minimum mean-square error in estimation, filtering and smoothing. IEEE Trans. Inform. Theory 51 1261–1282.
  • Kac, M., Murdock, W. L. and Szegö, G. (1953). On the eigenvalues of certain Hermitian forms. Indiana Univ. Math. J. (formerly J. Rational Mech. Anal.) 2 767–800.
  • Kadota, T. T., Zakai, M. and Ziv, J. (1971). Mutual information of the white Gaussian channel with and without feedback. IEEE Trans. Inform. Theory IT-17 368–371.
  • Lerer, L. E. (1978). On approximating the spectrum of convolution-type operators. I. Wiener–Hopf matricial integral operators. Israel J. Math. 30 339–362.
  • Pinsker, M. S. (1964). Information and Information Stability of Random Variables and Processes. Holden-Day, San Francisco, CA.
  • Snyders, J. (1972). Error formulae for optimal linear filtering, prediction and interpolation of stationary time series. Ann. Math. Statist. 43 1935–1943.
  • Snyders, J. (1972). Error expressions for optimal linear filtering of stationary processes. IEEE Trans. Inform. Theory IT-18 574–582.
  • Üstünel, A. S. and Zakai, M. (2000). Transformation of Measure on Wiener Space. Springer, Berlin.
  • Üstünel, A. S. and Zakai, M. (1997). The construction of filtrations on abstract Wiener space. Func. Anal. 143 10–32.
  • Yovits, M. C. and Jackson, J. L. (1955). Linear filter optimization with game theory considerations. In IRE Nat. Conv. Rec. 4 193–199.
  • Zakai, M. (2005). On mutual information, likelihood ratios, and estimation error for the additive Gaussian channel. IEEE Trans. Inform. Theory 51 3017–3024.