A Bernstein-type inequality for suprema of random processes with applications to model selection in non-Gaussian regression

Abstract

Let $(X_t)_{t∈T}$ be a family of real-valued centered random variables indexed by a countable set $T$. In the first part of this paper, we establish exponential bounds for the deviation probabilities of the supremum $Z = \sup_{t∈T} X_t$ by using the generic chaining device introduced in Talagrand (Inst. Hautes Études Sci. Publ. Math. 81 (1995) 73–205). Compared to concentration-type inequalities, these bounds offer the advantage of holding under weaker conditions on the family $(X_t)_{t∈T}$. The second part of the paper is oriented toward statistics. We consider the regression setting $Y = f + ξ$, where $f$ is an unknown vector in $ℝ^n$ and $ξ$ is a random vector, the components of which are independent, centered and admit finite Laplace transforms in a neighborhood of $0$. Our aim is to estimate $f$ from the observation of $Y$ by means of a model selection approach among a collection of linear subspaces of $ℝ^n$. The selection procedure we propose is based on the minimization of a penalized criterion, the penalty of which is calibrated by using the deviation bounds established in the first part of this paper. More precisely, we study suprema of random variables of the form $X_t = ∑_{i=1}^n t_iξ_i$, where $t$ varies in the unit ball of a linear subspace of $ℝ^n$. Finally, we show that our estimator satisfies an oracle-type inequality under suitable assumptions on the metric structures of the linear spaces of the collection.