Screening-based Bregman divergence estimation with NP-dimensionality

Abstract

Feature screening via the marginal screening ([5]; [7]) has gained special attention for high dimensional regression problems. However, their results are confined to the generalized linear model ($\mathrm{GLM}$) with the exponential family of distributions. This inspires us to explore the suitability of applying screening procedures to more general models, for example without assuming either the explicit form of distributions or parametric forms between response and covariates. In this paper, we extend the marginal screening procedure, by means of Bregman divergence (${\mathrm{BD}}$) as the loss function, to include not only the $\mathrm{GLM}$ but also the quasi-likelihood model. A sure screening property for the resulting screening procedure is established under this very general framework, assuming only certain moment conditions and tail properties, where the dimensionality $p_{n}$ is allowed to grow with the sample size $n$ as fast as $\log(p_{n})=O(n^{a})$ for some $a\in(0,1)$. Simulation and real data studies illustrate that a two-step procedure, which combines the feature screening in the first step and a penalized-${\mathrm{BD}}$ estimation in the second step, is practically applicable to identifying the set of relevant variables and achieving good estimation of model parameters, with the computational cost much less than those without using the screening step.