The Annals of Probability

Uniform convergence of Vapnik–Chervonenkis classes under ergodic sampling

Terrence M. Adams and Andrew B. Nobel

Full-text: Open access


We show that if $\mathcal{X}$ is a complete separable metric space and $\mathcal{C}$ is a countable family of Borel subsets of $\mathcal{X}$ with finite VC dimension, then, for every stationary ergodic process with values in $\mathcal{X}$, the relative frequencies of sets $C\in\mathcal{C}$ converge uniformly to their limiting probabilities. Beyond ergodicity, no assumptions are imposed on the sampling process, and no regularity conditions are imposed on the elements of $\mathcal{C}$. The result extends existing work of Vapnik and Chervonenkis, among others, who have studied uniform convergence for i.i.d. and strongly mixing processes. Our method of proof is new and direct: it does not rely on symmetrization techniques, probability inequalities or mixing conditions. The uniform convergence of relative frequencies for VC-major and VC-graph classes of functions under ergodic sampling is established as a corollary of the basic result for sets.

Article information

Ann. Probab., Volume 38, Number 4 (2010), 1345-1367.

First available in Project Euclid: 8 July 2010

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60F15: Strong theorems 37A50: Relations with probability theory and stochastic processes [See also 60Fxx and 60G10] 60C05: Combinatorial probability
Secondary: 60G10: Stationary processes 37A30: Ergodic theorems, spectral theory, Markov operators {For operator ergodic theory, see mainly 47A35}

VC dimension VC class ergodic process uniform convergence uniform law of large numbers


Adams, Terrence M.; Nobel, Andrew B. Uniform convergence of Vapnik–Chervonenkis classes under ergodic sampling. Ann. Probab. 38 (2010), no. 4, 1345--1367. doi:10.1214/09-AOP511.

Export citation


  • [1] Andrews, D. W. K. (1987). Consistency in nonlinear econometric models: A generic uniform law of large numbers. Econometrica 55 1465–1471.
  • [2] Billingsley, P. (1995). Probability and Measure, 3rd ed. Wiley, New York.
  • [3] Bradley, R. C. (1986). Basic properties of strong mixing conditions. In Dependence in Probability and Statistics (Oberwolfach, 1985). Progress in Probability 11 165–192. Birkhäuser, Boston, MA.
  • [4] Breiman, L. (1992). Probability. Classics in Applied Mathematics 7. SIAM, Philadelphia, PA.
  • [5] Devroye, L. and Lugosi, G. (2001). Combinatorial Methods in Density Estimation. Springer, New York.
  • [6] Dudley, R. M. (1999). Uniform Central Limit Theorems. Cambridge Studies in Advanced Mathematics 63. Cambridge Univ. Press, Cambridge.
  • [7] Hoffmann-Jørgensen, J. (1985). Necessary and sufficient condition for the uniform law of large numbers. In Probability in Banach Spaces, V (Medford, Mass., 1984). Lecture Notes in Math. 1153 258–272. Springer, Berlin.
  • [8] Karandikar, R. L. and Vidyasagar, M. (2002). Rates of uniform convergence of empirical means with mixing processes. Statist. Probab. Lett. 58 297–307.
  • [9] Matoušek, J. (2002). Lectures on Discrete Geometry. Graduate Texts in Mathematics 212. Springer, New York.
  • [10] Nobel, A. (1995). A counterexample concerning uniform ergodic theorems for a class of functions. Statist. Probab. Lett. 24 165–168.
  • [11] Nobel, A. and Dembo, A. (1993). A note on uniform laws of averages for dependent processes. Statist. Probab. Lett. 17 169–172.
  • [12] Parthasarathy, K. R. (1967). Probability Measures on Metric Spaces. Probability and Mathematical Statistics 3. Academic Press, New York.
  • [13] Peligrad, M. (2001). A note on the uniform laws for dependent processes via coupling. J. Theoret. Probab. 14 979–988.
  • [14] Petersen, K. (1983). Ergodic Theory. Cambridge Studies in Advanced Mathematics 2. Cambridge Univ. Press, Cambridge.
  • [15] Peškir, G. (1998). The uniform mean-square ergodic theorem for wide sense stationary processes. Stoch. Anal. Appl. 16 697–720.
  • [16] Peškir, G. and Weber, M. (1994). Necessary and sufficient conditions for the uniform law of large numbers in the stationary case. In Functional Analysis, IV (Dubrovnik, 1993). Various Publications Series 43 165–190. Aarhus Univ., Aarhus.
  • [17] Peškir, G. and Yukich, J. E. (1994). Uniform ergodic theorems for dynamical systems under VC entropy conditions. In Probability in Banach Spaces, 9 (Sandjberg, 1993). Progress in Probability 35 105–128. Birkhäuser, Boston, MA.
  • [18] Pollard, D. (1984). Convergence of Stochastic Processes. Springer, New York.
  • [19] Royden, H. L. (1988). Real Analysis, 3rd ed. Macmillan Co., New York.
  • [20] Sauer, N. (1972). On the density of families of sets. J. Combin. Theory Ser. A 13 145–147.
  • [21] Steele, J. M. (1978). Empirical discrepancies and subadditive processes. Ann. Probab. 6 118–127.
  • [22] Talagrand, M. (1987). The Glivenko–Cantelli problem. Ann. Probab. 15 837–870.
  • [23] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York.
  • [24] Vapnik, V. N. and Chervonenkis, A. Y. (1971). On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16 264–280.
  • [25] Vapnik, V. N. and Chervonenkis, A. Y. (1981). Necessary and sufficient conditions for the uniform convergence of means to their expectations. Theory Probab. Appl. 26 532–553.
  • [26] Yu, B. (1994). Rates of convergence for empirical processes of stationary mixing sequences. Ann. Probab. 22 94–116.
  • [27] Yukich, J. E. (1986). Rates of convergence for classes of functions: The non-i.i.d. case. J. Multivariate Anal. 20 175–189.