The Annals of Applied Probability

Some relations between mutual information and estimation error in Wiener space

Eddy Mayer-Wolf and Moshe Zakai

Full-text: Open access

Abstract

The model considered is that of “signal plus white noise.” Known connections between the noncausal filtering error and mutual information are combined with new ones involving the causal estimation error, in a general abstract setup. The results are shown to be invariant under a wide class of causality patterns; they are applied to the derivation of the causal estimation error of a Gaussian nonstationary filtering problem and to a multidimensional extension of the Yovits–Jackson formula.

Article information

Source
Ann. Appl. Probab. Volume 17, Number 3 (2007), 1102-1116.

Dates
First available: 22 May 2007

Permanent link to this document
http://projecteuclid.org/euclid.aoap/1179839183

Digital Object Identifier
doi:10.1214/105051607000000131

Mathematical Reviews number (MathSciNet)
MR2326241

Zentralblatt MATH identifier
1125.60035

Subjects
Primary: 60G35: Signal detection and filtering [See also 62M20, 93E10, 93E11, 94Axx] 94A17: Measures of information, entropy
Secondary: 28C20: Set functions and measures and integrals in infinite-dimensional spaces (Wiener measure, Gaussian measure, etc.) [See also 46G12, 58C35, 58D20, 60B11] 60H35: Computational methods for stochastic equations [See also 65C30]

Keywords
Estimation errors mutual information abstract Wiener space resolution of identity causality

Citation

Mayer-Wolf, Eddy; Zakai, Moshe. Some relations between mutual information and estimation error in Wiener space. The Annals of Applied Probability 17 (2007), no. 3, 1102--1116. doi:10.1214/105051607000000131. http://projecteuclid.org/euclid.aoap/1179839183.


Export citation

References

  • Duncan, T. E. (1970). On the calculation of mutual information. SIAM J. Appl. Math. 19 215--220.
  • Grenander, U. and Szegö, G. (1958). Toeplitz Forms and Their Applications. Univ. California Press, Berkeley.
  • Guo, D., Shamai, S. and Verdú, S. (2005). Mutual information and minimum mean-square error in estimation, filtering and smoothing. IEEE Trans. Inform. Theory 51 1261--1282.
  • Kac, M., Murdock, W. L. and Szegö, G. (1953). On the eigenvalues of certain Hermitian forms. Indiana Univ. Math. J. (formerly J. Rational Mech. Anal.) 2 767--800.
  • Kadota, T. T., Zakai, M. and Ziv, J. (1971). Mutual information of the white Gaussian channel with and without feedback. IEEE Trans. Inform. Theory IT-17 368--371.
  • Lerer, L. E. (1978). On approximating the spectrum of convolution-type operators. I. Wiener--Hopf matricial integral operators. Israel J. Math. 30 339--362.
  • Pinsker, M. S. (1964). Information and Information Stability of Random Variables and Processes. Holden-Day, San Francisco, CA.
  • Snyders, J. (1972). Error formulae for optimal linear filtering, prediction and interpolation of stationary time series. Ann. Math. Statist. 43 1935--1943.
  • Snyders, J. (1972). Error expressions for optimal linear filtering of stationary processes. IEEE Trans. Inform. Theory IT-18 574--582.
  • Üstünel, A. S. and Zakai, M. (2000). Transformation of Measure on Wiener Space. Springer, Berlin.
  • Üstünel, A. S. and Zakai, M. (1997). The construction of filtrations on abstract Wiener space. Func. Anal. 143 10--32.
  • Yovits, M. C. and Jackson, J. L. (1955). Linear filter optimization with game theory considerations. In IRE Nat. Conv. Rec. 4 193--199.
  • Zakai, M. (2005). On mutual information, likelihood ratios, and estimation error for the additive Gaussian channel. IEEE Trans. Inform. Theory 51 3017--3024.