The Annals of Statistics

The fused Kolmogorov filter: A nonparametric model-free screening method

Qing Mai and Hui Zou

Full-text: Open access


A new model-free screening method called the fused Kolmogorov filter is proposed for high-dimensional data analysis. This new method is fully nonparametric and can work with many types of covariates and response variables, including continuous, discrete and categorical variables. We apply the fused Kolmogorov filter to deal with variable screening problems emerging from a wide range of applications, such as multiclass classification, nonparametric regression and Poisson regression, among others. It is shown that the fused Kolmogorov filter enjoys the sure screening property under weak regularity conditions that are much milder than those required for many existing nonparametric screening methods. In particular, the fused Kolmogorov filter can still be powerful when covariates are strongly dependent on each other. We further demonstrate the superior performance of the fused Kolmogorov filter over existing screening methods by simulations and real data examples.

Article information

Ann. Statist., Volume 43, Number 4 (2015), 1471-1497.

Received: October 2014
First available in Project Euclid: 17 June 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G99: None of the above, but in this section

Variable screening high-dimensional data sure screening property


Mai, Qing; Zou, Hui. The fused Kolmogorov filter: A nonparametric model-free screening method. Ann. Statist. 43 (2015), no. 4, 1471--1497. doi:10.1214/14-AOS1303.

Export citation


  • Breiman, L. and Friedman, J. H. (1985). Estimating optimal transformations for multiple regression and correlation. J. Amer. Statist. Assoc. 80 580–619.
  • Chang, J., Tang, C. Y. and Wu, Y. (2013). Marginal empirical likelihood and sure independence feature screening. Ann. Statist. 41 2123–2148.
  • Cook, R. D. and Weisberg, S. (1991). Comment on “Sliced inverse regression for dimension reduction.” J. Amer. Statist. Assoc. 86 328–332.
  • Cook, R. D. and Zhang, X. (2014). Fused estimators of the central subspace in sufficient dimension reduction. J. Amer. Statist. Assoc. 109 815–827.
  • Fan, J. and Fan, Y. (2008). High-dimensional classification using features annealed independence rules. Ann. Statist. 36 2605–2637.
  • Fan, J., Feng, Y. and Song, R. (2011). Nonparametric independence screening in sparse ultra-high-dimensional additive models. J. Amer. Statist. Assoc. 106 544–557.
  • Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360.
  • Fan, J. and Lv, J. (2008). Sure independence screening for ultra-high dimensional feature space. J. Roy. Statist. Soc. Ser. B 20 101–148.
  • Fan, J., Samworth, R. and Wu, Y. (2009). Ultra-dimensional variable selection via independenct learning: Beyond the linear model. J. Mach. Learn. Res. 10 1829–1853.
  • Fan, J. and Song, R. (2010). Sure independence screening in generalized linear models with NP-dimensionality. Ann. Statist. 38 3567–3604.
  • Hastie, T. and Tibshirani, R. (1998). Classification by pairwise coupling. Ann. Statist. 26 451–471.
  • Hastie, T., Tibshirani, R. and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. Springer Series in Statistics. Springer, New York.
  • He, X. and Shen, L. (1997). Linear regression after spline transformation. Biometrika 84 474–481.
  • He, X., Wang, L. and Hong, H. G. (2013). Quantile–adaptive model-free variable screening for high-dimensional heterogeneous data. Ann. Statist. 41 342–369.
  • Hsing, T. and Carroll, R. J. (1992). An asymptotic theory for sliced inverse regression. Ann. Statist. 20 1040–1061.
  • Huang, J., Horowitz, J. L. and Ma, S. (2008). Asymptotic properties of bridge estimators in sparse high-dimensional regression models. Ann. Statist. 36 587–613.
  • Li, K.-C. (1991). Sliced inverse regression for dimension reduction. J. Amer. Statist. Assoc. 86 316–342.
  • Li, R., Zhong, W. and Zhu, L. (2012). Feature screening via distance correlation learning. J. Amer. Statist. Assoc. 107 1129–1139.
  • Li, G., Peng, H., Zhang, J. and Zhu, L. (2012). Robust rank correlation based screening. Ann. Statist. 40 1846–1877.
  • Lv, J. and Fan, Y. (2009). A unified approach to model selection and sparse recovery using regularized least squares. Ann. Statist. 37 3498–3528.
  • Mai, Q. and Zou, H. (2013). The Kolmogorov filter for variable screening in high-dimensional binary classification. Biometrika 100 229–234.
  • Székely, G. J., Rizzo, M. L. and Bakirov, N. K. (2007). Measuring and testing dependence by correlation of distances. Ann. Statist. 35 2769–2794.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267–288.
  • Venables, W. N. and Ripley, B. D. (2002). Modern Applied Statistics with S, 4th ed. Springer, New York.
  • Yang, Y. and Zou, H. (2015). A fast unified algorithm for solving group-lasso penalized learning problems. Stat. Comput. To appear. DOI:10.1007/s11222-014-9498-5.
  • Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty. Ann. Statist. 38 894–942.
  • Zhu, L. X. and Ng, K. W. (1995). Asymptotics of sliced inverse regression. Statist. Sinica 5 727–736.
  • Zhu, L.-P., Li, L., Li, R. and Zhu, L.-X. (2011). Model-free feature screening for ultrahigh-dimensional data. J. Amer. Statist. Assoc. 106 1464–1475.