Moment inequalities for functions of independent random variables

Moment inequalities for functions of independent random variables

Stéphane Boucheron, Olivier Bousquet, Gábor Lugosi, and Pascal Massart

Source: Ann. Probab. Volume 33, Number 2 (2005), 514-560.

Abstract

A general method for obtaining moment inequalities for functions of independent random variables is presented. It is a generalization of the entropy method which has been used to derive concentration inequalities for such functions [Boucheron, Lugosi and Massart Ann. Probab. 31 (2003) 1583–1614], and is based on a generalized tensorization inequality due to Latała and Oleszkiewicz [Lecture Notes in Math. 1745 (2000) 147–168]. The new inequalities prove to be a versatile tool in a wide range of applications. We illustrate the power of the method by showing how it can be used to effortlessly re-derive classical inequalities including Rosenthal and Kahane–Khinchine-type inequalities for sums of independent random variables, moment inequalities for suprema of empirical processes and moment inequalities for Rademacher chaos and U-statistics. Some of these corollaries are apparently new. In particular, we generalize Talagrand’s exponential inequality for Rademacher chaos of order 2 to any order. We also discuss applications for other complex functions of independent random variables, such as suprema of Boolean polynomials which include, as special cases, subgraph counting problems in random graphs.

Primary Subjects: 60E15, 60C05, 28A35

Secondary Subjects: 05C80

Keywords: Moment inequalities; concentration inequalities; empirical processes; random graphs

Full-text: Open access

Screen Optimized PDF File (291 KB)

Links and Identifiers

Permanent link to this document: http://projecteuclid.org/euclid.aop/1109868590
Digital Object Identifier: doi:10.1214/009117904000000856
Mathematical Reviews number (MathSciNet): MR2123200
Zentralblatt MATH identifier: 02164472

back to Table of Contents

References

Bartlett, P., Boucheron, S. and Lugosi, G. (2001). Model selection and error estimation. Machine Learning 48 85--113.

Bartlett, P., Bousquet, O. and Mendelson, S. (2002). Localized Rademacher complexities. Computational Learning Theory. Lecture Notes in Comput. Sci. 2375 44--58. Springer, Berlin.

Mathematical Reviews (MathSciNet): MR2040404

Zentralblatt MATH: 1050.68054

Bartlett, P. L. and Mendelson, S. (2002). Rademacher and Gaussian complexities: Risk bounds and structural results. J. Mach. Learn. Res. 3 463--482.

Mathematical Reviews (MathSciNet): MR1984026

Digital Object Identifier: doi:10.1162/153244303321897690

Beckner, W. (1989). A generalized Poincaré inequality for Gaussian measures. Proc. Amer. Math. Soc. 105 397--400.

Mathematical Reviews (MathSciNet): MR954373

Bonami, A. (1970). Étude des coefficients de Fourier des fonctions de $l^p(g)$. Ann. Inst. Fourier (Grenoble) 20 335--402.

Mathematical Reviews (MathSciNet): MR283496

Boucheron, S., Lugosi, G. and Massart, P. (2000). A sharp concentration inequality with applications. Random Structures Algorithms 16 277--292.

Mathematical Reviews (MathSciNet): MR1749290

Boucheron, S., Lugosi, G. and Massart, P. (2003). Concentration inequalities using the entropy method. Ann. Probab. 31 1583--1614.

Mathematical Reviews (MathSciNet): MR1989444

Digital Object Identifier: doi:10.1214/aop/1055425791

Project Euclid: euclid.aop/1055425791

Bousquet, O. (2002). A Bennett concentration inequality and its application to suprema of empirical processes. C. R. Acad. Sci. Paris Sér. I 334 495--500.

Mathematical Reviews (MathSciNet): MR1890640

Digital Object Identifier: doi:10.1016/S1631-073X(02)02292-6

Burkholder, D. L. (1988). Sharp inequalities for martingales and stochastic integrals. Astérisque 157--158 75--94.

Mathematical Reviews (MathSciNet): MR976214

Burkholder, D. L. (1989). Explorations in martingale theory and its applications. Ecole d'Eté de Probabilités de Saint-Flour XIX. Lecture Notes in Math. 1464 1--66. Springer, New York.

Mathematical Reviews (MathSciNet): MR1108183

Zentralblatt MATH: 0771.60033

Chafaï, D. (2002). Entropies, convexity, and functional inequalities. Technical report. Available at arxiv.org/abs/math.PR/0211103.

Chow, Y. S. and Teicher, H. (1978). Probability Theory, Independence, Interchangeability, Martingales. Springer, New York.

Mathematical Reviews (MathSciNet): MR513230

Zentralblatt MATH: 0399.60001

de la Peña, V. H. and Giné, E. (1999). Decoupling: From Dependence to Independence. Springer, New York.

Mathematical Reviews (MathSciNet): MR1666908

Devroye, L. (2002). Laws of large numbers and tail inequalities for random tries and Patricia trees. J. Comput. Appl. Math. 142 27--37.

Mathematical Reviews (MathSciNet): MR1910516

Digital Object Identifier: doi:10.1016/S0377-0427(01)00458-7

Efron, B. and Stein, C. (1981). The jackknife estimate of variance. Ann. Statist. 9 586--596.

Mathematical Reviews (MathSciNet): MR615434

JSTOR: links.jstor.org

Giné, E., Latała, R. and Zinn, J. (2000). Exponential and moment inequalities for $u$-statistics. In High Dimensional Probability II 13--38. Birkhäuser, Boston.

Janson, S. (1990). Poisson approximation for large deviations. Random Structures Algorithms 1 221--230.

Mathematical Reviews (MathSciNet): MR1138428

Janson, S., Łuczak, T. and Ruciński, A. (2000). Random Graphs. Wiley, New York.

Janson, S., Oleszkiewicz, K. and Ruciński, A. (2004). Upper tails for subgraph counts in random graphs. Israel J. Math. To appear.

Mathematical Reviews (MathSciNet): MR2085711

Janson, S. and Ruciński, A. (2000). The deletion method for upper tail estimates. Unpublished manuscript.

Kim, J. H. and Vu, V. (2004). Divide and conquer martingales and the number of triangles in a random graph. Random Structures Algorithms 24 166--174.

Mathematical Reviews (MathSciNet): MR2035874

Koltchinskii, V. (2001). Rademacher penalties and structural risk minimization. IEEE Trans. Inform. Theory 47 1902--1914.

Mathematical Reviews (MathSciNet): MR1842526

Digital Object Identifier: doi:10.1109/18.930926

Koltchinskii, V. and Panchenko, D. (2000). Rademacher processes and bounding the risk of function learning. In High Dimensional Probability II (E. Giné, D. Mason and J. Wellner, eds.) 443--459. Birkhäuser, Boston.

Mathematical Reviews (MathSciNet): MR1857339

Zentralblatt MATH: 01552503

Kwapień, S. and Woyczyńsky, W. (1992). Random Series and Stochastic Integrals: Single and Multiple. Birkhäuser, Boston.

Mathematical Reviews (MathSciNet): MR1167198

Latała, R. and Oleszkiewicz, C. (2000). Between Sobolev and Poincaré. Geometric Aspects of Functional Analysis, Israel Seminar (GAFA) 1996--2000. Lecture Notes in Math. 1745 147--168. Springer, Berlin.

Ledoux, M. (1997). On Talagrand's deviation inequalities for product measures. ESAIM Probab. Statist. 1 63--87.

Mathematical Reviews (MathSciNet): MR1399224

Ledoux, M. (1999). Concentration of measure and logarithmic Sobolev inequalities. Séminaire de Probabilités XXXIII. Lecture Notes in Math. 1709 120--216. Springer, New York.

Mathematical Reviews (MathSciNet): MR1767995

Zentralblatt MATH: 0957.60016

Ledoux, M. (2001). The Concentration of Measure Phenomenon. Amer. Math. Soc., Providence, RI.

Mathematical Reviews (MathSciNet): MR1849347

Zentralblatt MATH: 0995.60002

Ledoux, M. and Talagrand, M. (1991). Probability in Banach Space. Springer, New York.

Mathematical Reviews (MathSciNet): MR1102015

Zentralblatt MATH: 0748.60004

Massart, P. (2000). About the constants in Talagrand's concentration inequalities for empirical processes. Ann. Probab. 28 863--884.

Mathematical Reviews (MathSciNet): MR1782276

Digital Object Identifier: doi:10.1214/aop/1019160263

Project Euclid: euclid.aop/1019160263

Massart, P. (2000). Some applications of concentration inequalities to statistics. Ann. Fac. Sci. Toulouse Math. (6) 9 245--303.

Mathematical Reviews (MathSciNet): MR1813803

McDiarmid, C. (1989). On the method of bounded differences. In Surveys in Combinatorics 1989 (J. Siemons, ed.) 148--188. Cambridge Univ. Press.

Mathematical Reviews (MathSciNet): MR1036755

Zentralblatt MATH: 0712.05012

McDiarmid, C. (1998). Concentration. In Probabilistic Methods for Algorithmic Discrete Mathematics (M. Habib, C. McDiarmid, J. Ramirez-Alfonsin and B. Reed, eds.) 195--248. Springer, New York.

Mathematical Reviews (MathSciNet): MR1678578

Zentralblatt MATH: 0927.60027

Pinelis, I. (1994). Optimum bounds for the distributions of martingales in Banach spaces. Ann. Probab. 22 1679--1706.

Mathematical Reviews (MathSciNet): MR1331198

JSTOR: links.jstor.org

Pinelis, I. (1995). Optimum bounds on moments of sums of independent random vectors. Siberian Adv. Math. 5 141--150.

Mathematical Reviews (MathSciNet): MR1387858

Rio, E. (2001). Inégalités de concentration pour les processus empiriques de classes de parties. Probab. Theory Related Fields 119 163--175.

Mathematical Reviews (MathSciNet): MR1818244

Digital Object Identifier: doi:10.1007/PL00008756

Steele, J. M. (1986). An Efron--Stein inequality for nonsymmetric statistics. Ann. Statist. 14 753--758.

Mathematical Reviews (MathSciNet): MR840528

JSTOR: links.jstor.org

Suen, W. C. S. (1990). A correlation inequality and a Poisson limit theorem for nonoverlapping balanced subgraphs of a random graph. Random Structures Algorithms 1 231--242.

Mathematical Reviews (MathSciNet): MR1138429

Talagrand, M. (1995). Concentration of measure and isoperimetric inequalities in product spaces. Inst. Hautes Études Sci. Publ. Math. 81 73--205.

Mathematical Reviews (MathSciNet): MR1361756

Talagrand, M. (1996). New concentration inequalities in product spaces. Invent. Math. 126 505--563.

Mathematical Reviews (MathSciNet): MR1419006

Digital Object Identifier: doi:10.1007/s002220050108

Talagrand, M. (1996). A new look at independence. Ann. Probab. 24 1--34.

Mathematical Reviews (MathSciNet): MR1387624

Digital Object Identifier: doi:10.1214/aop/1065725175

Project Euclid: euclid.aop/1042644705

previous :: next