The Annals of Statistics

The fused Kolmogorov filter: A nonparametric model-free screening method

Qing Mai and Hui Zou

Full-text: Open access

Abstract

A new model-free screening method called the fused Kolmogorov filter is proposed for high-dimensional data analysis. This new method is fully nonparametric and can work with many types of covariates and response variables, including continuous, discrete and categorical variables. We apply the fused Kolmogorov filter to deal with variable screening problems emerging from a wide range of applications, such as multiclass classification, nonparametric regression and Poisson regression, among others. It is shown that the fused Kolmogorov filter enjoys the sure screening property under weak regularity conditions that are much milder than those required for many existing nonparametric screening methods. In particular, the fused Kolmogorov filter can still be powerful when covariates are strongly dependent on each other. We further demonstrate the superior performance of the fused Kolmogorov filter over existing screening methods by simulations and real data examples.

Article information

Source
Ann. Statist., Volume 43, Number 4 (2015), 1471-1497.

Dates
Received: October 2014
First available in Project Euclid: 17 June 2015

Permanent link to this document
https://projecteuclid.org/euclid.aos/1434546212

Digital Object Identifier
doi:10.1214/14-AOS1303

Mathematical Reviews number (MathSciNet)
MR3357868

Zentralblatt MATH identifier
06470427

Subjects
Primary: 62G99: None of the above, but in this section

Keywords
Variable screening high-dimensional data sure screening property

Citation

Mai, Qing; Zou, Hui. The fused Kolmogorov filter: A nonparametric model-free screening method. Ann. Statist. 43 (2015), no. 4, 1471--1497. doi:10.1214/14-AOS1303. https://projecteuclid.org/euclid.aos/1434546212


Export citation

References

  • Breiman, L. and Friedman, J. H. (1985). Estimating optimal transformations for multiple regression and correlation. J. Amer. Statist. Assoc. 80 580–619.
  • Chang, J., Tang, C. Y. and Wu, Y. (2013). Marginal empirical likelihood and sure independence feature screening. Ann. Statist. 41 2123–2148.
  • Cook, R. D. and Weisberg, S. (1991). Comment on “Sliced inverse regression for dimension reduction.” J. Amer. Statist. Assoc. 86 328–332.
  • Cook, R. D. and Zhang, X. (2014). Fused estimators of the central subspace in sufficient dimension reduction. J. Amer. Statist. Assoc. 109 815–827.
  • Fan, J. and Fan, Y. (2008). High-dimensional classification using features annealed independence rules. Ann. Statist. 36 2605–2637.
  • Fan, J., Feng, Y. and Song, R. (2011). Nonparametric independence screening in sparse ultra-high-dimensional additive models. J. Amer. Statist. Assoc. 106 544–557.
  • Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360.
  • Fan, J. and Lv, J. (2008). Sure independence screening for ultra-high dimensional feature space. J. Roy. Statist. Soc. Ser. B 20 101–148.
  • Fan, J., Samworth, R. and Wu, Y. (2009). Ultra-dimensional variable selection via independenct learning: Beyond the linear model. J. Mach. Learn. Res. 10 1829–1853.
  • Fan, J. and Song, R. (2010). Sure independence screening in generalized linear models with NP-dimensionality. Ann. Statist. 38 3567–3604.
  • Hastie, T. and Tibshirani, R. (1998). Classification by pairwise coupling. Ann. Statist. 26 451–471.
  • Hastie, T., Tibshirani, R. and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. Springer Series in Statistics. Springer, New York.
  • He, X. and Shen, L. (1997). Linear regression after spline transformation. Biometrika 84 474–481.
  • He, X., Wang, L. and Hong, H. G. (2013). Quantile–adaptive model-free variable screening for high-dimensional heterogeneous data. Ann. Statist. 41 342–369.
  • Hsing, T. and Carroll, R. J. (1992). An asymptotic theory for sliced inverse regression. Ann. Statist. 20 1040–1061.
  • Huang, J., Horowitz, J. L. and Ma, S. (2008). Asymptotic properties of bridge estimators in sparse high-dimensional regression models. Ann. Statist. 36 587–613.
  • Li, K.-C. (1991). Sliced inverse regression for dimension reduction. J. Amer. Statist. Assoc. 86 316–342.
  • Li, R., Zhong, W. and Zhu, L. (2012). Feature screening via distance correlation learning. J. Amer. Statist. Assoc. 107 1129–1139.
  • Li, G., Peng, H., Zhang, J. and Zhu, L. (2012). Robust rank correlation based screening. Ann. Statist. 40 1846–1877.
  • Lv, J. and Fan, Y. (2009). A unified approach to model selection and sparse recovery using regularized least squares. Ann. Statist. 37 3498–3528.
  • Mai, Q. and Zou, H. (2013). The Kolmogorov filter for variable screening in high-dimensional binary classification. Biometrika 100 229–234.
  • Székely, G. J., Rizzo, M. L. and Bakirov, N. K. (2007). Measuring and testing dependence by correlation of distances. Ann. Statist. 35 2769–2794.
  • Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267–288.
  • Venables, W. N. and Ripley, B. D. (2002). Modern Applied Statistics with S, 4th ed. Springer, New York.
  • Yang, Y. and Zou, H. (2015). A fast unified algorithm for solving group-lasso penalized learning problems. Stat. Comput. To appear. DOI:10.1007/s11222-014-9498-5.
  • Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty. Ann. Statist. 38 894–942.
  • Zhu, L. X. and Ng, K. W. (1995). Asymptotics of sliced inverse regression. Statist. Sinica 5 727–736.
  • Zhu, L.-P., Li, L., Li, R. and Zhu, L.-X. (2011). Model-free feature screening for ultrahigh-dimensional data. J. Amer. Statist. Assoc. 106 1464–1475.