The Annals of Probability

On Hoeffding’s inequalities

Vidmantas Bentkus

Abstract

In a celebrated work by Hoeffding [ J. Amer. Statist. Assoc. 58 (1963) 13–30], several inequalities for tail probabilities of sums Mn={X}1++{X}n of bounded independent random variables Xj were proved. These inequalities had a considerable impact on the development of probability and statistics, and remained unimproved until 1995 when Talagrand [Inst. Hautes Études Sci. Publ. Math. 81 (1995a) 73–205] inserted certain missing factors in the bounds of two theorems. By similar factors, a third theorem was refined by Pinelis [Progress in Probability 43 (1998) 257–314] and refined (and extended) by me. In this article, I introduce a new type of inequality. Namely, I show that ℙ{Mnx}c{Snx}, where c is an absolute constant and Sn={ɛ}1++{ɛ}n is a sum of independent identically distributed Bernoulli random variables (a random variable is called Bernoulli if it assumes at most two values). The inequality holds for those xℝ where the survival function x{Snx} has a jump down. For the remaining x the inequality still holds provided that the function between the adjacent jump points is interpolated linearly or log-linearly. If it is necessary, to estimate ℙ{Snx} special bounds can be used for binomial probabilities. The results extend to martingales with bounded differences. It is apparent that Theorem 1.1 of this article is the most important. The inequalities have applications to measure concentration, leading to results of the type where, up to an absolute constant, the measure concentration is dominated by the concentration in a simplest appropriate model, such results will be considered elsewhere.

Article information

Source
Ann. Probab. Volume 32, Number 2 (2004), 1650-1673.

Dates
First available in Project Euclid: 18 May 2004

http://projecteuclid.org/euclid.aop/1084884866

Digital Object Identifier
doi:10.1214/009117904000000360

Mathematical Reviews number (MathSciNet)
MR2060313

Zentralblatt MATH identifier
1062.60011

Subjects
Primary: 60E15: Inequalities; stochastic orderings

Citation

Bentkus, Vidmantas. On Hoeffding’s inequalities. Ann. Probab. 32 (2004), no. 2, 1650--1673. doi:10.1214/009117904000000360. http://projecteuclid.org/euclid.aop/1084884866.

References

• Alon, N. and Milman, V. D. (1984). Concentration of measure phenomena in the discrete case and the Laplace operator of a graph. Publ. Math. Univ. Paris VII 20 55--68.
• Bahadur, R. R. (1971). Some Limit Theorems in Statistics. SIAM, Philadelphia.
• Bentkus, V. (1994). On the asymptotical behavior of the constant in the Berry--Esseen inequality. J. Theoret. Probab. 7 211--224.
• Bentkus, V. (2001). An inequality for large deviation probabilities of sums of bounded i.i.d. r.v. Lithuanian Math. J. 41 144--153.
• Bentkus, V. (2003). An inequality for tail probabilities of martingales with differences bounded from one side. J. Theor. Probab. 16 161--173.
• Bentkus, V. (2004). On measure concentration for separately Lipschitz functions in product spaces. Israel J. Math. To appear.
• Bentkus, V. and Kirsha, K. (1989). Estimates for the closeness of a distribution function to the normal law. Lithuanian Math. J. 29 321--332.
• Bentkus, V. and van Zuijlen, M. (2003). Conservative confidence bounds for the mean Lithuanian Math. J. 43 141--160.
• Bretagnolle, J. (1980). Statistique de Kolmogorov--Smirnov pour un enchantillon nonequireparti. Colloqvium Internationale CNRS 307 39--44.
• Eaton, M. L. (1970). A note on symmetric Bernoulli random variables. Ann. Math. Statist. 41 1223--1226.
• Eaton, M. L. (1974). A probability inequality for linear combinations of bounded random variables. Ann. Statist. 2 609--613.
• Gleser, L. J. (1975). On the distribution of the number of successes in independent trials. Ann. Probab. 3 182--188.
• Gromov, M. and Milman, V. D. (1983). A topological application of the isoperimetric inequality. Amer. J. Math. 105 843--854.
• Hoeffding, W. (1956). On the distribution of the number of successes in independent trials. Ann. Math. Statist. 27 713--721.
• Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 13--30.
• Ibragimov, I. A. (1956). On the composition of unimodal distributions. Theory Probab. Appl. 1 255--260.
• Karlin, S. and Studden, W. J. (1966). Tchebycheff Systems: With Applications in Analysis and Statistics. Wiley, New York.
• Keilson, J. and Gerber, H. (1971). Some results for discrete unimodality. J. Amer. Statist. Assoc. 66 386--389.
• Ledoux, M. (1999). Concentration of measure and logarithmic Sobolev inequalities. Séminaire de Probabilités XXXIII. Lecture Notes in Math. 1709 120--216. Springer, Berlin.
• Marshall, A. W. and Olkin, I. (1979). Inequalities: Theory of Majorization and Its Applications. Academic Press, New York.
• McDiarmid, C. (1989). On the method of bounded differences. In Surveys in Combinatorics. London Math. Soc. Lecture Note Ser. 141 148--188. Cambridge Univ. Press.
• Milman, V. D. (1985). Random subspaces of proportional dimension of finite-dimensional normed spaces: Approach through the isoperimetric inequality. Lecture Notes in Math. 1166 106--115. Springer, Berlin.
• Milman, V. D. (1988). The heritage of P. Lévy in geometrical functional analysis. Astérisque 157--158 273--301.
• Milman, V. D. and Schechtman, G. (1986). Asymptotic theory of finite-dimensional normed spaces. Lecture Notes in Math. 1200. Springer, Berlin.
• Ostrowski, A. (1952). Sur quelques applications des fonctions convexes et concave au sens de I. Schur. J. Math. Pures Appl. 31 253--292.
• Paulauskas, V. (2002). Some comments on deviation probabilities for infinitely divisible random vectors. Lithuanian Math. J. 42 394--410.
• Petrov, V. V. (1975). Sums of Independent Random Variables. Springer, New York.
• Pinelis, I. (1994). Extremal probabilistic problems and Hotelling's $T^2$ test under a symmetry assumption. Ann. Statist. 22 357--368.
• Pinelis, I. (1998). Optimal tail comparison based on comparison of moments. In High Dimensional Probability. Progress in Probability 43 297--314. Birkhäuser, Basel.
• Pinelis, I. (1999). Fractional sums and integrals of $r$-concave tails and applications to comparison probability inequalities. In Advances in Stochastic Inequalities. Contemporary Mathematics 234 149--168. Amer. Math. Soc. Providence, RI.
• Schur, I. (1923). Über eine Klasse von Mittelbildungen mit Anwendungen auf die Determinantentheorie. Sitzungsber. Berlin Math. Ges. 22 9--20.
• Shorack, G. R. and Wellner, J. A. (1986). Empirical Processes with Applications to Statistics. Wiley, New York.
• Talagrand, M. (1995a). Concentration of measure and isoperimetric inequalities in product spaces. Inst. Hautes Études Sci. Publ. Math. 81 73--205.
• Talagrand, M. (1995b). The missing factor in Hoeffding's inequalities. Ann. Inst. H. Poincaré Probab. Statist. 31 689--702.