The Annals of Statistics
- Ann. Statist.
- Volume 43, Number 4 (2015), 1498-1534.
QUADRO: A supervised dimension reduction method via Rayleigh quotient optimization
We propose a novel Rayleigh quotient based sparse quadratic dimension reduction method—named QUADRO ( Quadratic Dimension Reduction via Rayleigh Optimization)—for analyzing high-dimensional data. Unlike in the linear setting where Rayleigh quotient optimization coincides with classification, these two problems are very different under nonlinear settings. In this paper, we clarify this difference and show that Rayleigh quotient optimization may be of independent scientific interests. One major challenge of Rayleigh quotient optimization is that the variance of quadratic statistics involves all fourth cross-moments of predictors, which are infeasible to compute for high-dimensional applications and may accumulate too many stochastic errors. This issue is resolved by considering a family of elliptical models. Moreover, for heavy-tail distributions, robust estimates of mean vectors and covariance matrices are employed to guarantee uniform convergence in estimating nonpolynomially many parameters, even though only the fourth moments are assumed. Methodologically, QUADRO is based on elliptical models which allow us to formulate the Rayleigh quotient maximization as a convex optimization problem. Computationally, we propose an efficient linearized augmented Lagrangian method to solve the constrained optimization problem. Theoretically, we provide explicit rates of convergence in terms of Rayleigh quotient under both Gaussian and general elliptical models. Thorough numerical results on both synthetic and real datasets are also provided to back up our theoretical results.
Ann. Statist., Volume 43, Number 4 (2015), 1498-1534.
Received: November 2013
Revised: December 2014
First available in Project Euclid: 17 June 2015
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 62H30: Classification and discrimination; cluster analysis [See also 68T10, 91C20]
Secondary: 62G20: Asymptotic properties
Fan, Jianqing; Ke, Zheng Tracy; Liu, Han; Xia, Lucy. QUADRO: A supervised dimension reduction method via Rayleigh quotient optimization. Ann. Statist. 43 (2015), no. 4, 1498--1534. doi:10.1214/14-AOS1307. https://projecteuclid.org/euclid.aos/1434546213
- Supplement to “QUADRO: A supervised dimension reduction method via Rayleigh quotient optimization”. Owing to space constraints, numerical tables for simulation and some of the technical proofs are relegated to a supplementary document. It contains proofs of Propositions 2.1, 5.1 and 6.2.