Bounding $\bar{d}$-distance by informational divergence: a method to prove measure concentration

K. Marton

doi:10.1214/aop/1039639365

April 1996 Bounding $\bar{d}$-distance by informational divergence: a method to prove measure concentration

K. Marton

Ann. Probab. 24(2): 857-866 (April 1996). DOI: 10.1214/aop/1039639365

Abstract

There is a simple inequality by Pinsker between variational distance and informational divergence of probability measures defined on arbitrary probability spaces. We shall consider probability measures on sequences taken from countable alphabets, and derive, from Pinsker's inequality, bounds on the $\bar{d}$-distance by informational divergence. Such bounds can be used to prove the "concentration of measure" phenomenon for some nonproduct distributions.

References

1.

AHLSWEDE, R., GACS, P. and KORNER, J. 1976. Bounds on conditional probabilities with applica´ ¨ tions in multi-user communication. Z. Wahrsch. Verw. Gebiete 34 157 177. Z. MR57:11927 10.1007/BF00535682AHLSWEDE, R., GACS, P. and KORNER, J. 1976. Bounds on conditional probabilities with applica´ ¨ tions in multi-user communication. Z. Wahrsch. Verw. Gebiete 34 157 177. Z. MR57:11927 10.1007/BF00535682

2.

CSISZAR, I. and KORNER, J. 1981. Information Theory: Coding Theorems for Discrete Memory´ ¨ less Sy stems. Academic Press, New York. Z. CSISZAR, I. and KORNER, J. 1981. Information Theory: Coding Theorems for Discrete Memory´ ¨ less Sy stems. Academic Press, New York. Z.

3.

MARTON, K. 1986. A simple proof of the blowing-up lemma. IEEE Trans. Inform. Theory IT-32 445 446. Z. MR87e:94018 10.1109/TIT.1986.1057176MARTON, K. 1986. A simple proof of the blowing-up lemma. IEEE Trans. Inform. Theory IT-32 445 446. Z. MR87e:94018 10.1109/TIT.1986.1057176

4.

MARTON, K. 1995a. A concentration-of-measure inequality for contracting Markov chains. Geometric and Functional Analy sis. To appear. Z. MARTON, K. 1995a. A concentration-of-measure inequality for contracting Markov chains. Geometric and Functional Analy sis. To appear. Z.

5.

MARTON, K. 1995b. Processes having the blowing-up property. Unpublished manuscript. Z. MARTON, K. 1995b. Processes having the blowing-up property. Unpublished manuscript. Z.

6.

MARTON, K. and SHIELDS, P. C. 1994. The positive divergence and blowing-up properties. Israeli J. Math. 86 331 348. Z. MR96g:28022 0797.60044 10.1007/BF02773685MARTON, K. and SHIELDS, P. C. 1994. The positive divergence and blowing-up properties. Israeli J. Math. 86 331 348. Z. MR96g:28022 0797.60044 10.1007/BF02773685

7.

MCDIARMID, C. 1989. On the method of bounded differences. In Survey s in Combinatorics. Z. London Mathematical Society Lecture Notes J. Simons, ed. 141 148 188. Cambridge Univ. Press, London. Z. MR91e:05077 0712.05012MCDIARMID, C. 1989. On the method of bounded differences. In Survey s in Combinatorics. Z. London Mathematical Society Lecture Notes J. Simons, ed. 141 148 188. Cambridge Univ. Press, London. Z. MR91e:05077 0712.05012

8.

PAPAMARCOU, A. and SHALABY, H. 1993. Error exponent for distributed detection of Markov sources. IEEE Trans. Inform. Theory 40 397 408. Z. PAPAMARCOU, A. and SHALABY, H. 1993. Error exponent for distributed detection of Markov sources. IEEE Trans. Inform. Theory 40 397 408. Z.

9.

PINSKER, M. S. 1964. Information and Information Stability of Random Variables and Processes. Holden-Day, San Francisco. Z. MR35:4054bPINSKER, M. S. 1964. Information and Information Stability of Random Variables and Processes. Holden-Day, San Francisco. Z. MR35:4054b

10.

TALAGRAND, M. 1995. Concentration of measure and isoperimetric inequalities in product spaces. Publ. IHES. 81 73 205. MR97h:60016 10.1007/BF02699376TALAGRAND, M. 1995. Concentration of measure and isoperimetric inequalities in product spaces. Publ. IHES. 81 73 205. MR97h:60016 10.1007/BF02699376

Citation Download Citation

K. Marton "Bounding $\bar{d}$-distance by informational divergence: a method to prove measure concentration," The Annals of Probability 24(2), 857-866, (April 1996). https://doi.org/10.1214/aop/1039639365

Published: April 1996

Access the abstract

JOURNAL ARTICLE
10 PAGES

DOWNLOAD PDF + SAVE TO MY LIBRARY