Annals of Probability
- Ann. Probab.
- Volume 34, Number 3 (2006), 1143-1216.
Concentration inequalities and asymptotic results for ratio type empirical processes
Evarist Giné and Vladimir Koltchinskii
Abstract
Let ℱ be a class of measurable functions on a measurable space $(S,\mathscr{S})$ with values in [0,1] and let $$P_n=n^{−1}\sum_{i=1}^nδ{X_i}$$ be the empirical measure based on an i.i.d. sample (X1,…,Xn) from a probability distribution P on $(S,\mathscr{S})$. We study the behavior of suprema of the following type: $$\sup_{r_{n}<\sigma_{P}f\leq \delta_{n}}\frac{|P_{n}f-Pf|}{\phi(\sigma_{P}f)},$$ where σPf≥Var1/2Pf and ϕ is a continuous, strictly increasing function with ϕ(0)=0. Using Talagrand’s concentration inequality for empirical processes, we establish concentration inequalities for such suprema and use them to derive several results about their asymptotic behavior, expressing the conditions in terms of expectations of localized suprema of empirical processes. We also prove new bounds for expected values of sup-norms of empirical processes in terms of the largest σPf and the L2(P) norm of the envelope of the function class, which are especially suited for estimating localized suprema. With this technique, we extend to function classes most of the known results on ratio type suprema of empirical processes, including some of Alexander’s results for VC classes of sets. We also consider applications of these results to several important problems in nonparametric statistics and in learning theory (including general excess risk bounds in empirical risk minimization and their versions for L2-regression and classification and ratio type bounds for margin distributions in classification).
Article information
Source
Ann. Probab., Volume 34, Number 3 (2006), 1143-1216.
Dates
First available in Project Euclid: 27 June 2006
Permanent link to this document
https://projecteuclid.org/euclid.aop/1151418495
Digital Object Identifier
doi:10.1214/009117906000000070
Mathematical Reviews number (MathSciNet)
MR2243881
Zentralblatt MATH identifier
1152.60021
Subjects
Primary: 60E15: Inequalities; stochastic orderings
Secondary: 60F17: Functional limit theorems; invariance principles 60F15: Strong theorems 62G08: Nonparametric regression 68T10: Pattern recognition, speech recognition {For cluster analysis, see 62H30}
Keywords
Ratio type empirical processes concentration inequalities ratio limit theorems localized sup-norms weighted central limit theorems VC type classes moment bounds for empirical processes nonparametric regression classification
Citation
Giné, Evarist; Koltchinskii, Vladimir. Concentration inequalities and asymptotic results for ratio type empirical processes. Ann. Probab. 34 (2006), no. 3, 1143--1216. doi:10.1214/009117906000000070. https://projecteuclid.org/euclid.aop/1151418495
