The Annals of Probability

Moment inequalities for functions of independent random variables

Stéphane Boucheron, Olivier Bousquet, Gábor Lugosi, and Pascal Massart

Full-text: Open access

Abstract

A general method for obtaining moment inequalities for functions of independent random variables is presented. It is a generalization of the entropy method which has been used to derive concentration inequalities for such functions [Boucheron, Lugosi and Massart Ann. Probab. 31 (2003) 1583–1614], and is based on a generalized tensorization inequality due to Latała and Oleszkiewicz [Lecture Notes in Math. 1745 (2000) 147–168]. The new inequalities prove to be a versatile tool in a wide range of applications. We illustrate the power of the method by showing how it can be used to effortlessly re-derive classical inequalities including Rosenthal and Kahane–Khinchine-type inequalities for sums of independent random variables, moment inequalities for suprema of empirical processes and moment inequalities for Rademacher chaos and U-statistics. Some of these corollaries are apparently new. In particular, we generalize Talagrand’s exponential inequality for Rademacher chaos of order 2 to any order. We also discuss applications for other complex functions of independent random variables, such as suprema of Boolean polynomials which include, as special cases, subgraph counting problems in random graphs.

Article information

Source
Ann. Probab., Volume 33, Number 2 (2005), 514-560.

Dates
First available in Project Euclid: 3 March 2005

Permanent link to this document
https://projecteuclid.org/euclid.aop/1109868590

Digital Object Identifier
doi:10.1214/009117904000000856

Mathematical Reviews number (MathSciNet)
MR2123200

Zentralblatt MATH identifier
1074.60018

Subjects
Primary: 60E15: Inequalities; stochastic orderings 60C05: Combinatorial probability 28A35: Measures and integrals in product spaces
Secondary: 05C80: Random graphs [See also 60B20]

Keywords
Moment inequalities concentration inequalities empirical processes random graphs

Citation

Boucheron, Stéphane; Bousquet, Olivier; Lugosi, Gábor; Massart, Pascal. Moment inequalities for functions of independent random variables. Ann. Probab. 33 (2005), no. 2, 514--560. doi:10.1214/009117904000000856. https://projecteuclid.org/euclid.aop/1109868590


Export citation

References

  • Bartlett, P., Boucheron, S. and Lugosi, G. (2001). Model selection and error estimation. Machine Learning 48 85--113.
  • Bartlett, P., Bousquet, O. and Mendelson, S. (2002). Localized Rademacher complexities. Computational Learning Theory. Lecture Notes in Comput. Sci. 2375 44--58. Springer, Berlin.
  • Bartlett, P. L. and Mendelson, S. (2002). Rademacher and Gaussian complexities: Risk bounds and structural results. J. Mach. Learn. Res. 3 463--482.
  • Beckner, W. (1989). A generalized Poincaré inequality for Gaussian measures. Proc. Amer. Math. Soc. 105 397--400.
  • Bonami, A. (1970). Étude des coefficients de Fourier des fonctions de $l^p(g)$. Ann. Inst. Fourier (Grenoble) 20 335--402.
  • Boucheron, S., Lugosi, G. and Massart, P. (2000). A sharp concentration inequality with applications. Random Structures Algorithms 16 277--292.
  • Boucheron, S., Lugosi, G. and Massart, P. (2003). Concentration inequalities using the entropy method. Ann. Probab. 31 1583--1614.
  • Bousquet, O. (2002). A Bennett concentration inequality and its application to suprema of empirical processes. C. R. Acad. Sci. Paris Sér. I 334 495--500.
  • Burkholder, D. L. (1988). Sharp inequalities for martingales and stochastic integrals. Astérisque 157--158 75--94.
  • Burkholder, D. L. (1989). Explorations in martingale theory and its applications. Ecole d'Eté de Probabilités de Saint-Flour XIX. Lecture Notes in Math. 1464 1--66. Springer, New York.
  • Chafaï, D. (2002). Entropies, convexity, and functional inequalities. Technical report. Available at arxiv.org/abs/math.PR/0211103.
  • Chow, Y. S. and Teicher, H. (1978). Probability Theory, Independence, Interchangeability, Martingales. Springer, New York.
  • de la Peña, V. H. and Giné, E. (1999). Decoupling: From Dependence to Independence. Springer, New York.
  • Devroye, L. (2002). Laws of large numbers and tail inequalities for random tries and Patricia trees. J. Comput. Appl. Math. 142 27--37.
  • Efron, B. and Stein, C. (1981). The jackknife estimate of variance. Ann. Statist. 9 586--596.
  • Giné, E., Latała, R. and Zinn, J. (2000). Exponential and moment inequalities for $u$-statistics. In High Dimensional Probability II 13--38. Birkhäuser, Boston.
  • Janson, S. (1990). Poisson approximation for large deviations. Random Structures Algorithms 1 221--230.
  • Janson, S., Łuczak, T. and Ruciński, A. (2000). Random Graphs. Wiley, New York.
  • Janson, S., Oleszkiewicz, K. and Ruciński, A. (2004). Upper tails for subgraph counts in random graphs. Israel J. Math. To appear.
  • Janson, S. and Ruciński, A. (2000). The deletion method for upper tail estimates. Unpublished manuscript.
  • Kim, J. H. and Vu, V. (2004). Divide and conquer martingales and the number of triangles in a random graph. Random Structures Algorithms 24 166--174.
  • Koltchinskii, V. (2001). Rademacher penalties and structural risk minimization. IEEE Trans. Inform. Theory 47 1902--1914.
  • Koltchinskii, V. and Panchenko, D. (2000). Rademacher processes and bounding the risk of function learning. In High Dimensional Probability II (E. Giné, D. Mason and J. Wellner, eds.) 443--459. Birkhäuser, Boston.
  • Kwapień, S. and Woyczyńsky, W. (1992). Random Series and Stochastic Integrals: Single and Multiple. Birkhäuser, Boston.
  • Latała, R. and Oleszkiewicz, C. (2000). Between Sobolev and Poincaré. Geometric Aspects of Functional Analysis, Israel Seminar (GAFA) 1996--2000. Lecture Notes in Math. 1745 147--168. Springer, Berlin.
  • Ledoux, M. (1997). On Talagrand's deviation inequalities for product measures. ESAIM Probab. Statist. 1 63--87.
  • Ledoux, M. (1999). Concentration of measure and logarithmic Sobolev inequalities. Séminaire de Probabilités XXXIII. Lecture Notes in Math. 1709 120--216. Springer, New York.
  • Ledoux, M. (2001). The Concentration of Measure Phenomenon. Amer. Math. Soc., Providence, RI.
  • Ledoux, M. and Talagrand, M. (1991). Probability in Banach Space. Springer, New York.
  • Massart, P. (2000). About the constants in Talagrand's concentration inequalities for empirical processes. Ann. Probab. 28 863--884.
  • Massart, P. (2000). Some applications of concentration inequalities to statistics. Ann. Fac. Sci. Toulouse Math. (6) 9 245--303.
  • McDiarmid, C. (1989). On the method of bounded differences. In Surveys in Combinatorics 1989 (J. Siemons, ed.) 148--188. Cambridge Univ. Press.
  • McDiarmid, C. (1998). Concentration. In Probabilistic Methods for Algorithmic Discrete Mathematics (M. Habib, C. McDiarmid, J. Ramirez-Alfonsin and B. Reed, eds.) 195--248. Springer, New York.
  • Pinelis, I. (1994). Optimum bounds for the distributions of martingales in Banach spaces. Ann. Probab. 22 1679--1706.
  • Pinelis, I. (1995). Optimum bounds on moments of sums of independent random vectors. Siberian Adv. Math. 5 141--150.
  • Rio, E. (2001). Inégalités de concentration pour les processus empiriques de classes de parties. Probab. Theory Related Fields 119 163--175.
  • Steele, J. M. (1986). An Efron--Stein inequality for nonsymmetric statistics. Ann. Statist. 14 753--758.
  • Suen, W. C. S. (1990). A correlation inequality and a Poisson limit theorem for nonoverlapping balanced subgraphs of a random graph. Random Structures Algorithms 1 231--242.
  • Talagrand, M. (1995). Concentration of measure and isoperimetric inequalities in product spaces. Inst. Hautes Études Sci. Publ. Math. 81 73--205.
  • Talagrand, M. (1996). New concentration inequalities in product spaces. Invent. Math. 126 505--563.
  • Talagrand, M. (1996). A new look at independence. Ann. Probab. 24 1--34.