Electronic Journal of Statistics

Significance testing in non-sparse high-dimensional linear models

Yinchu Zhu and Jelena Bradic

Full-text: Open access

Enhanced PDF (638 KB)

Abstract

In high-dimensional linear models, the sparsity assumption is typically made, stating that most of the parameters are equal to zero. Under the sparsity assumption, estimation and, recently, inference have been well studied. However, in practice, sparsity assumption is not checkable and more importantly is often violated; a large number of covariates might be expected to be associated with the response, indicating that possibly all, rather than just a few, parameters are non-zero. A natural example is a genome-wide gene expression profiling, where all genes are believed to affect a common disease marker. We show that existing inferential methods are sensitive to the sparsity assumption, and may, in turn, result in the severe lack of control of Type-I error. In this article, we propose a new inferential method, named CorrT, which is robust to model misspecification such as heteroscedasticity and lack of sparsity. CorrT is shown to have Type I error approaching the nominal level for any models and Type II error approaching zero for sparse and many dense models. In fact, CorrT is also shown to be optimal in a variety of frameworks: sparse, non-sparse and hybrid models where sparse and dense signals are mixed. Numerical experiments show a favorable performance of the CorrT test compared to the state-of-the-art methods.

Article information

Source
Electron. J. Statist., Volume 12, Number 2 (2018), 3312-3364.

Dates
Received: January 2017
First available in Project Euclid: 6 October 2018

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1538791404

Digital Object Identifier
doi:10.1214/18-EJS1443

Rights
Creative Commons Attribution 4.0 International License.

Citation

Zhu, Yinchu; Bradic, Jelena. Significance testing in non-sparse high-dimensional linear models. Electron. J. Statist. 12 (2018), no. 2, 3312--3364. doi:10.1214/18-EJS1443. https://projecteuclid.org/euclid.ejs/1538791404


Export citation

References

  • Acion, L., Kelmansky, D., van der Laan, M., Sahker, E., Jones, D. and Arndt, S. (2017). Use of a machine learning framework to predict substance use disorder treatment success., PloS one 12 e0175383.
  • Arriaga, J. M., Bravo, A. I., Mordoh, J. and Bianchini, M. (2017). Metallothionein 1G promotes the differentiation of HT-29 human colorectal cancer cells., Oncology Reports 37 2633–2651.
  • Belloni, A., Chernozhukov, V. and Wang, L. (2011). Square-root lasso: pivotal recovery of sparse signals via conic programming., Biometrika 98 791–806.
    Zentralblatt MATH: 1228.62083
    Digital Object Identifier: doi:10.1093/biomet/asr043
  • Belloni, A., Chernozhukov, V. and Hansen, C. (2014). Inference on treatment effects after selection among high-dimensional controls., The Review of Economic Studies 81 608–650.
  • Belloni, A., Chernozhukov, V. and Kato, K. (2018). Valid post-selection inference in high-dimensional approximately sparse quantile regression models., Journal of the American Statistical Association just-accepted 1–33.
  • Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous analysis of Lasso and Dantzig selector., The Annals of Statistics 37 1705–1732.
    Zentralblatt MATH: 1173.62022
    Digital Object Identifier: doi:10.1214/08-AOS620
    Project Euclid: euclid.aos/1245332830
  • Borovkov, A. A. (2000). Estimates for the distribution of sums and maxima of sums of random variables without the Cramer condition., Siberian Mathematical Journal 41 811–848.
    Mathematical Reviews (MathSciNet): MR1803562
  • Bosse, K., Haneder, S., Arlt, C., Ihling, C. H., Seufferlein, T. and Sinz, A. (2016). Mass spectrometry-based secretome analysis of non-small cell lung cancer cell lines., Proteomics 16 2801–2814.
  • Boucheron, S., Lugosi, G. and Massart, P. (2013)., Concentration inequalities: A nonasymptotic theory of independence. Oxford university press.
    Zentralblatt MATH: 1279.60005
  • Bradic, J., Fan, J. and Zhu, Y. (2018). Testability of high-dimensional linear models with non-sparse structures., arXiv preprint arXiv:1802.09117.
  • Bühlmann, P. and Van de Geer, S. (2011)., Statistics for high-dimensional data: methods, theory and applications. Springer Science & Business Media.
    Mathematical Reviews (MathSciNet): MR2807761
  • Cai, T. T., Guo, Z. et al. (2017). Confidence intervals for high-dimensional linear regression: Minimax rates and adaptivity., The Annals of statistics 45 615–646.
    Zentralblatt MATH: 1371.62045
    Digital Object Identifier: doi:10.1214/16-AOS1461
    Project Euclid: euclid.aos/1494921952
  • Cavalier, L. and Tsybakov, A. (2002). Sharp adaptation for inverse problems with random noise., Probability Theory and Related Fields 123 323–354.
    Zentralblatt MATH: 1039.62031
    Digital Object Identifier: doi:10.1007/s004400100169
  • Chernozhukov, V., Hansen, C. and Spindler, M. (2015). Valid post-selection and post-regularization inference: An elementary, general approach., Annual Review of Economics.
  • Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C. and Newey, W. (2017a). Double/Debiased/Neyman machine learning of treatment effects., American Economic Review 107 261–65.
  • Chernozhukov, V., Hansen, C., Liao, Y. et al. (2017b). A lava attack on the recovery of sums of dense and sparse signals., The Annals of Statistics 45 39–76.
    Zentralblatt MATH: 06710505
    Digital Object Identifier: doi:10.1214/16-AOS1434
    Project Euclid: euclid.aos/1487667617
  • Collier, O., Comminges, L. and Tsybakov, A. B. (2018). Some effects in adaptive robust estimation under sparsity., arXiv preprint arXiv:1802.04230.
    Zentralblatt MATH: 06968611
    Digital Object Identifier: doi:10.1214/17-AOS1653
    Project Euclid: euclid.aos/1536307245
  • Collier, O., Comminges, L., Tsybakov, A. B. and Verzelen, N. (2016). Optimal adaptive estimation of linear functionals under sparsity., arXiv preprint arXiv:1611.09744.
    Zentralblatt MATH: 06968611
    Digital Object Identifier: doi:10.1214/17-AOS1653
    Project Euclid: euclid.aos/1536307245
  • Dicker, L. H. (2016). Ridge regression and asymptotic minimax estimation over spheres of growing dimension., Bernoulli 22 1–37.
    Zentralblatt MATH: 1388.62205
    Digital Object Identifier: doi:10.3150/14-BEJ609
    Project Euclid: euclid.bj/1443620842
  • Donoho, D. L. and Johnstone, I. M. (1994). Minimax risk over lp -balls for lq -error., Probability Theory and Related Fields 99 277–303.
    Zentralblatt MATH: 0802.62006
    Digital Object Identifier: doi:10.1007/BF01199026
  • Donoho, D. L. and Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage., Journal of the American Statistical Association 90 1200–1224.
    Zentralblatt MATH: 0869.62024
    Digital Object Identifier: doi:10.1080/01621459.1995.10476626
  • Ellis, M. J., Jenkins, S., Hanfelt, J., Redington, M. E., Taylor, M., Leek, R., Siddle, K. and Harris, A. (1998). Insulin-like growth factors in human breast cancer., Breast Cancer Research and Treatment 52 175–184.
  • Feller, W. (1968)., An introduction to probability theory and its applications: volume I 3. John Wiley & Sons London-New York-Sydney-Toronto.
    Zentralblatt MATH: 0155.23101
  • Gautier, E. and Tsybakov, A. B. (2013). Pivotal estimation in high-dimensional regression via linear programming. In, Empirical Inference 195–204. Springer.
    Zentralblatt MATH: 1325.62088
  • Hall, P. and Heyde, C. C. (1980)., Martingale limit theory and its application. Academic press New York.
    Zentralblatt MATH: 0462.60045
  • Harvey, A. C. (1976). Estimating regression models with multiplicative heteroscedasticity., Econometrica 461–465.
    Zentralblatt MATH: 0333.62040
    Digital Object Identifier: doi:10.2307/1913974
  • Holm, K., Staaf, J., Jönsson, G., Vallon-Christersson, J., Gunnarsson, H., Arason, A., Magnusson, L., Barkardottir, R. B., Hegardt, C., Ringnér, M. and Borg, Å. (2012). Characterisation of amplification patterns and target genes at chromosome 11q13 in CCND1-amplified sporadic and familial breast tumours., Breast Cancer Research and Treatment 133 583–594.
  • Imbens, G. W. (2004). Nonparametric estimation of average treatment effects under exogeneity: A review., Review of Economics and statistics 86 4–29.
  • Imbens, G. W. and Wooldridge, J. M. (2009). Recent developments in the econometrics of program evaluation., Journal of Economic Literature 47 5–86.
  • Ingster, Y. I., Tsybakov, A. B. and Verzelen, N. (2010). Detection boundary in sparse regression., Electronic Journal of Statistics 4 1476–1526.
    Zentralblatt MATH: 1329.62314
    Digital Object Identifier: doi:10.1214/10-EJS589
  • Janson, L., Barber, R. F. and Candes, E. (2017). EigenPrism: inference for high dimensional signal-to-noise ratios., Journal of the Royal Statistical Society: Series B (Statistical Methodology) 79 1037–1065.
    Zentralblatt MATH: 1373.62355
    Digital Object Identifier: doi:10.1111/rssb.12203
  • Javanmard, A. and Montanari, A. (2014a). Confidence intervals and hypothesis testing for high-dimensional regression., The Journal of Machine Learning Research 15 2869–2909.
    Zentralblatt MATH: 1319.62145
  • Javanmard, A. and Montanari, A. (2014b). Hypothesis testing in high-dimensional regression under the gaussian random design model: Asymptotic theory., IEEE Transactions on Information Theory 60 6522–6554.
    Zentralblatt MATH: 1360.62074
    Digital Object Identifier: doi:10.1109/TIT.2014.2343629
  • Javanmard, A. and Montanari, A. (2018). De-biasing the Lasso: Optimal Sample Size for Gaussian Designs., forthcoming in The Annals of Statistics.
    Zentralblatt MATH: 06968593
    Digital Object Identifier: doi:10.1214/17-AOS1630
    Project Euclid: euclid.aos/1536307227
  • Kitange, G., Mladek, A., Schroeder, M., Pokorny, J., Carlson, B., Zhang, Y., Nair, A., Lee, J.-H., Yan, H., Decker, P., Zhang, Z. and Sarkaria, J. (2016). Retinoblastoma Binding Protein 4 Modulates Temozolomide Sensitivity in Glioblastoma by Regulating DNA Repair Proteins., Cell Reports 14 2587–2598.
  • Lee, S., Wu, M. C. and Lin, X. (2012). Optimal tests for rare variant effects in sequencing association studies., Biostatistics 13 762–775.
  • Lehmann, E. L. and Romano, J. P. (2006)., Testing statistical hypotheses. Springer Science & Business Media.
    Mathematical Reviews (MathSciNet): MR2135927
    Zentralblatt MATH: 1076.62018
  • Li, H., Lee, T.-H. and Avraham, H. (2002). A Novel Tricomplex of BRCA1, Nmi, and c-Myc Inhibits c-Myc-induced Human Telomerase Reverse Transcriptase Gene (hTERT) Promoter Activity in Breast Cancer., Journal of Biological Chemistry 277 20965–20973.
  • Liu, L., Miao, W., Sun, B., Robins, J. M. and Tchetgen Tchetgen, E. J. (2015). Doubly robust estimation of a marginal average effect of treatment on the treated with an instrumental variable., Harvard Working Paper Series.
  • Ma, Y. and Zhu, L. (2013). Doubly robust and efficient estimators for heteroscedastic partially linear single-index models allowing high dimensional covariates., Journal of the Royal Statistical Society: Series B (Statistical Methodology) 75 305–322.
  • Merlevède, F., Peligrad, M. and Rio, E. (2011). A Bernstein type inequality and moderate deviations for weakly dependent sequences., Probability Theory and Related Fields 151 435–474.
    Zentralblatt MATH: 1242.60020
    Digital Object Identifier: doi:10.1007/s00440-010-0304-9
  • Neale, B. M., Rivas, M. A., Voight, B. F., Altshuler, D., Devlin, B., Orho-Melander, M., Kathiresan, S., Purcell, S. M., Roeder, K. and Daly, M. J. (2011). Testing for an unusual distribution of rare variants., PLoS genetics 7 e1001322.
  • Newey, W. K. (1994). The asymptotic variance of semiparametric estimators., Econometrica 62 1349–1382.
    Mathematical Reviews (MathSciNet): MR1303237
    Zentralblatt MATH: 0816.62034
    Digital Object Identifier: doi:10.2307/2951752
  • Neyman, J. (1959). Optimal asymptotic tests of composite statistical hypotheses., The Harald Cramer Volume, ed. by U. Grenander 213–234.
  • Ning, Y., Liu, H. et al. (2017). A general theory of hypothesis tests and confidence regions for sparse high dimensional models., The Annals of Statistics 45 158–195.
    Zentralblatt MATH: 1364.62128
    Digital Object Identifier: doi:10.1214/16-AOS1448
    Project Euclid: euclid.aos/1487667620
  • Oates, A. J., Schumaker, L. M., Jenkins, S. B., Pearce, A. A., DaCosta, S. A., Arun, B. and Ellis, M. J. (1998). The mannose 6-phosphate/insulin-like growth factor 2 receptor (M6P/IGF2R), a putative breast tumor suppressor gene., Breast cancer research and treatment 47 269–281.
  • Pang, H., Liu, H. and Vanderbei, R. J. (2014). The fastclime package for linear programming and large-scale precision matrix estimation in R., Journal of Machine Learning Research 15 489–493.
    Zentralblatt MATH: 1318.90002
  • Park, R. E. (1966). Estimation with heteroscedastic error terms., Econometrica 34 888.
  • Peña, V. H., Lai, T. L. and Shao, Q.-M. (2008)., Self-normalized processes: Limit theory and Statistical Applications. Springer Science & Business Media.
    Mathematical Reviews (MathSciNet): MR2488094
  • Poczobutt, J. M., Nguyen, T. T., Hanson, D., Li, H., Sippel, T. R., Weiser-Evans, M. C., Gijon, M., Murphy, R. C. and Nemenoff, R. A. (2016). Deletion of 5-lipoxygenase in the tumor microenvironment promotes lung cancer progression and metastasis through regulating T cell recruitment., The Journal of Immunology 196 891–901.
  • Raskutti, G., Wainwright, M. J. and Yu, B. (2011). Minimax rates of estimation for high-dimensional linear regression over-balls., IEEE Transactions on Information Theory 57 6976–6994.
    Zentralblatt MATH: 1365.62276
    Digital Object Identifier: doi:10.1109/TIT.2011.2165799
  • Robins, J. M., Rotnitzky, A. and Zhao, L. P. (1994). Estimation of regression coefficients when some regressors are not always observed., Journal of the American Statistical Association 89 846–866.
    Zentralblatt MATH: 0815.62043
    Digital Object Identifier: doi:10.1080/01621459.1994.10476818
  • Robins, J. M. and Rotnitzky, A. (1995). Semiparametric efficiency in multivariate regression models with missing data., Journal of the American Statistical Association 90 122–129.
    Zentralblatt MATH: 0818.62043
    Digital Object Identifier: doi:10.1080/01621459.1995.10476494
  • Robins, J. M., Rotnitzky, A. and Zhao, L. P. (1995). Analysis of semiparametric regression models for repeated outcomes in the presence of missing data., Journal of the American Statistical Association 90 106–121.
    Zentralblatt MATH: 0818.62042
    Digital Object Identifier: doi:10.1080/01621459.1995.10476493
  • Robins, J. M. and Rotnitzky, A. (2001). Comments., Statistica Sinica 920–936.
  • Rotnitzky, A., Robins, J. M. and Scharfstein, D. O. (1998). Semiparametric regression for repeated outcomes with nonignorable nonresponse., Journal of the American Statistical Association 93 1321–1339.
    Zentralblatt MATH: 1064.62520
    Digital Object Identifier: doi:10.1080/01621459.1998.10473795
  • Rubin, D. B. and van der Laan, M. J. (2008). Empirical efficiency maximization: Improved locally efficient covariate adjustment in randomized experiments and survival analysis., The International Journal of Biostatistics 4.
  • Rudelson, M. and Zhou, S. (2013). Reconstruction from anisotropic random measurements., IEEE Transactions on Information Theory 59 3434–3447.
    Zentralblatt MATH: 1364.94158
    Digital Object Identifier: doi:10.1109/TIT.2013.2243201
  • Sun, T. and Zhang, C.-H. (2012). Scaled sparse linear regression., Biometrika 99 879–898.
    Zentralblatt MATH: 06111558
    Digital Object Identifier: doi:10.1093/biomet/ass043
  • Tang, N.-Y., Chueh, F.-S., Yu, C.-C., Liao, C.-L., Lin, J.-J., Hsia, T.-C., Wu, K.-C., Liu, H.-C., Lu, K.-W. and Chung, J.-G. (2016). Benzyl isothiocyanate alters the gene expression with cell cycle regulation and cell death in human brain glioblastoma GBM 8401 cells., Oncology reports 35 2089–2096.
  • Tchetgen, E. J. T. and Shpitser, I. (2012). Semiparametric theory for causal mediation analysis: efficiency bounds, multiple robustness, and sensitivity analysis., The Annals of Statistics 40 1816.
    Zentralblatt MATH: 1257.62033
    Digital Object Identifier: doi:10.1214/12-AOS990
    Project Euclid: euclid.aos/1350394518
  • Terracciano, D., Ferro, M., Terreri, S., Lucarelli, G., D’Elia, C., Musi, G., de Cobelli, O., Mirone, V. and Cimmino, A. (2017). Urinary long non-coding RNAs in non-muscle invasive bladder cancer: new architects in cancer prognostic biomarkers., Translational Research.
  • Van de Geer, S., Bühlmann, P., Ritov, Y. and Dezeure, R. (2014). On asymptotically optimal confidence regions and tests for high-dimensional models., The Annals of Statistics 42 1166–1202.
    Zentralblatt MATH: 1305.62259
    Digital Object Identifier: doi:10.1214/14-AOS1221
    Project Euclid: euclid.aos/1403276911
  • Van der Laan, M. J. and Robins, J. M. (2003)., Unified methods for censored longitudinal data and causality. Springer Science & Business Media.
    Zentralblatt MATH: 1013.62034
  • Van der Vaart, A. W. (2000)., Asymptotic statistics 3. Cambridge university press.
  • Vanderbei, R. J. (2014)., Linear Programming: Foundations and Extensions. Springer.
    Zentralblatt MATH: 1299.90243
  • Vershynin, R. (2010). Introduction to the non-asymptotic analysis of random matrices., arXiv preprint arXiv:1011.3027.
  • Wang, Y., Han, R., Chen, Z., Fu, M., Kang, J., Li, K., Li, L., Chen, H. and He, Y. (2016). A transcriptional miRNA-gene network associated with lung adenocarcinoma metastasis based on the TCGA database., Oncology reports 35 2257–2269.
  • White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity., Econometrica 817–838.
    Zentralblatt MATH: 0459.62051
    Digital Object Identifier: doi:10.2307/1912934
  • Zhang, C.-H. and Zhang, S. S. (2014). Confidence intervals for low dimensional parameters in high dimensional linear models., Journal of the Royal Statistical Society: Series B (Statistical Methodology) 76 217–242.
  • Zhang, M., Gao, C., Yang, Y., Li, G., Dong, J., Ai, Y., Ma, Q. and Li, W. (2017). MiR-424 Promotes Non-Small Cell Lung Cancer Progression and Metastasis through Regulating the Tumor Suppressor Gene TNFAIP1., Cellular Physiology and Biochemistry 42 211–221.
  • Zhao, Y.-Q., Zeng, D., Laber, E. B., Song, R., Yuan, M. and Kosorok, M. R. (2014). Doubly robust learning for estimating individualized treatment with censored data., Biometrika 102 151–168.
    Zentralblatt MATH: 1345.62092
    Digital Object Identifier: doi:10.1093/biomet/asu050
  • Zhu, Y. and Bradic, J. (2017). Linear hypothesis testing in dense high-dimensional linear models., Journal of the American Statistical Association just-accepted.
    Zentralblatt MATH: 06833592
    Digital Object Identifier: doi:10.1007/s11749-017-0556-0

The Institute of Mathematical Statistics and the Bernoulli Society

Electronic Journal of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more