The Annals of Statistics
- Ann. Statist.
- Volume 45, Number 2 (2017), 708-743.
A Bernstein-type inequality for some mixing processes and dynamical systems with an application to learning
We establish a Bernstein-type inequality for a class of stochastic processes that includes the classical geometrically $\phi$-mixing processes, Rio’s generalization of these processes and many time-discrete dynamical systems. Modulo a logarithmic factor and some constants, our Bernstein-type inequality coincides with the classical Bernstein inequality for i.i.d. data. We further use this new Bernstein-type inequality to derive an oracle inequality for generic regularized empirical risk minimization algorithms and data generated by such processes. Applying this oracle inequality to support vector machines using the Gaussian kernels for binary classification, we obtain essentially the same rate as for i.i.d. processes, and for least squares and quantile regression; it turns out that the resulting learning rates match, up to some arbitrarily small extra term in the exponent, the optimal rates for i.i.d. processes.
Ann. Statist., Volume 45, Number 2 (2017), 708-743.
Received: March 2015
Revised: March 2016
First available in Project Euclid: 16 May 2017
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 60E15: Inequalities; stochastic orderings
Secondary: 60G10: Stationary processes 37D20: Uniformly hyperbolic systems (expanding, Anosov, Axiom A, etc.) 60F10: Large deviations 68T05: Learning and adaptive systems [See also 68Q32, 91E40] 62G08: Nonparametric regression 62M10: Time series, auto-correlation, regression, etc. [See also 91B84]
Hang, Hanyuan; Steinwart, Ingo. A Bernstein-type inequality for some mixing processes and dynamical systems with an application to learning. Ann. Statist. 45 (2017), no. 2, 708--743. doi:10.1214/16-AOS1465. https://projecteuclid.org/euclid.aos/1494921955
- Supplement to “A Bernstein-type inequality for some mixing processes and dynamical systems with an application to learning”. The supplement  contains an Appendix, in which we provide the proofs for Sections 2 and 4.