December 2021 VCSEL: Prioritizing SNP-set by penalized variance component selection
Juhyun Kim, Judong Shen, Anran Wang, Devan V. Mehrotra, Seyoon Ko, Jin J. Zhou, Hua Zhou
Author Affiliations +
Ann. Appl. Stat. 15(4): 1652-1672 (December 2021). DOI: 10.1214/21-AOAS1491


Single nucleotide polymorphism (SNP) set analysis aggregates both common and rare variants and tests for association between phenotype(s) of interest and a set. However, multiple SNP-sets, such as genes, pathways, or sliding windows are usually investigated across the whole genome in which all groups are tested separately, followed by multiple testing adjustments. We propose a novel method to prioritize SNP-sets in a joint multivariate variance component model. Each SNP-set corresponds to a variance component (or kernel), and model selection is achieved by incorporating either convex or nonconvex penalties. The uniqueness of this variance component selection framework, which we call VCSEL, is that it naturally encompasses multivariate traits (VCSEL-M) and SNP-set-treatment or -environment interactions (VCSEL-I). We devise an optimization algorithm scalable to many variance components, based on the majorization-minimization (MM) principle. Simulation studies demonstrate the superiority of our methods in model selection performance, as measured by the area under the precision-recall (PR) curve, compared to the commonly used marginal testing and group penalization methods. Finally, we apply our methods to a real pharmacogenomics study and a real whole exome sequencing study. Some top ranked genes by VCSEL are detected as insignificant by the marginal test methods which emphasizes formal inference of individual genes with a strict significance threshold. This provides alternative insights for biologists to prioritize follow-up studies and develop polygenic risk score models.

Funding Statement

This work is partially supported by National Institutes of Health (NIH) grants T32 HG02536, R01 HG006139 and R35 GM141798 and National Science Foundation Grant DMS-2054253.


Download Citation

Juhyun Kim. Judong Shen. Anran Wang. Devan V. Mehrotra. Seyoon Ko. Jin J. Zhou. Hua Zhou. "VCSEL: Prioritizing SNP-set by penalized variance component selection." Ann. Appl. Stat. 15 (4) 1652 - 1672, December 2021.


Received: 1 August 2020; Revised: 1 May 2021; Published: December 2021
First available in Project Euclid: 21 December 2021

MathSciNet: MR4355070
zbMATH: 1498.62230
Digital Object Identifier: 10.1214/21-AOAS1491

Keywords: group selection , majorization-minimization (MM) , multiple phenotypes , nonconvex penalties , penalized estimation , Rare variants , Restricted Maximum Likelihood (REML) , variance components model

Rights: Copyright © 2021 Institute of Mathematical Statistics


This article is only available to subscribers.
It is not available for individual sale.

Vol.15 • No. 4 • December 2021
Back to Top