Bernoulli

  • Bernoulli
  • Volume 18, Number 2 (2012), 434-454.

Thermodynamics and concentration

Andreas Maurer

Full-text: Open access

Abstract

We show that the thermal subadditivity of entropy provides a common basis to derive a strong form of the bounded difference inequality and related results as well as more recent inequalities applicable to convex Lipschitz functions, random symmetric matrices, shortest travelling salesmen paths and weakly self-bounding functions. We also give two new concentration inequalities.

Article information

Source
Bernoulli, Volume 18, Number 2 (2012), 434-454.

Dates
First available in Project Euclid: 16 April 2012

Permanent link to this document
https://projecteuclid.org/euclid.bj/1334580719

Digital Object Identifier
doi:10.3150/10-BEJ341

Mathematical Reviews number (MathSciNet)
MR2922456

Zentralblatt MATH identifier
1285.60036

Keywords
concentration entropy method tail bounds

Citation

Maurer, Andreas. Thermodynamics and concentration. Bernoulli 18 (2012), no. 2, 434--454. doi:10.3150/10-BEJ341. https://projecteuclid.org/euclid.bj/1334580719


Export citation

References

  • [1] Bernstein, S. (1946). Theory of Probability. Gastehizdal Publishing House, Moscow.
  • [2] Boltzmann, L. (1877). Ueber die Beziehung zwischen dem zweiten Hauptsatz der mechanischen Waermetheorie und der Wahrscheinlichkeitsrechnung respektive den Saetzen ueber das Waermegleichgewicht. Wiener Berichte 76 373–435.
  • [3] Boucheron, S., Lugosi, G. and Massart, P. (2003). Concentration inequalities using the entropy method. Ann. Probab. 31 1583–1614.
  • [4] Boucheron, S., Lugosi, G. and Massart, P. (2009). On concentration of self-bounding functions. Electron. J. Probab. 14 1884–1899.
  • [5] Bousquet, O. (2002). A Bennett concentration inequality and its application to suprema of empirical processes. C. R. Math. Acad. Sci. Paris 334 495–500.
  • [6] Chebychev, P. (1874). Sur les valeurs limites des intégrales. J. Math. Pures Appl. 19 157–160.
  • [7] Gibbs, J.W. (1902). Elementary Principles in Statistical Mechanics with Especial Reference to the Rational Foundation of Thermodynamics. Yale, CT: Yale Univ. Press.
  • [8] Gross, L. (1975). Logarithmic Sobolev inequalities. Amer. J. Math. 97 1061–1083.
  • [9] Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 13–30.
  • [10] Ledoux, M. (1996). On Talagrand’s deviation inequalities for product measures. ESAIM Probab. Statist. 1 63–87 (electronic).
  • [11] Ledoux, M. (2001). The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89. Providence, RI: Amer. Math. Soc.
  • [12] Lieb, E.H. (1975). Some convexity and subadditivity properties of entropy. Bull. Amer. Math. Soc. 81 1–13.
  • [13] Massart, P. (2000). About the constants in Talagrand’s concentration inequalities for empirical processes. Ann. Probab. 28 863–884.
  • [14] Massart, P. (2000). Some applications of concentration inequalities to statistics. Ann. Fac. Sci. Toulouse Math. (6) 9 245–303.
  • [15] Maurer, A. (2003). A bound on the deviation probability for sums of non-negative random variables. JIPAM. J. Inequal. Pure Appl. Math. 4 1–6 (electronic).
  • [16] Maurer, A. (2006). Concentration inequalities for functions of independent variables. Random Structures Algorithms 29 121–138.
  • [17] Maurer, A. (2010). Dominated concentration. Statist. Probab. Lett. 80 683–689.
  • [18] McAllester, D. and Ortiz, L. (2002). Concentration inequalities for the missing mass and for histogram rule error. J. Mach. Learn. Res. 4 895–911.
  • [19] McDiarmid, C. (1998). Concentration. In Probabilistic Methods for Algorithmic Discrete Mathematics. Algorithms Combin. 16 195–248. Berlin: Springer.
  • [20] Pinelis, I.S. and Utev, S.A. (1989). Sharp exponential estimates for sums of independent random variables. Theory Probab. Appl. 34 340–346.
  • [21] Steele, J.M. (1986). An Efron–Stein inequality for nonsymmetric statistics. Ann. Statist. 14 753–758.
  • [22] Talagrand, M. (1995). Concentration of measure and isoperimetric inequalities in product spaces. Publ. Math. Inst. Hautes Études Sci. 81 73–205.
  • [23] Talagrand, M. (1996). A new look at independence. Ann. Probab. 24 1–34.