Translator Disclaimer
June 2016 SLOPE is adaptive to unknown sparsity and asymptotically minimax
Weijie Su, Emmanuel Candès
Ann. Statist. 44(3): 1038-1068 (June 2016). DOI: 10.1214/15-AOS1397

Abstract

We consider high-dimensional sparse regression problems in which we observe $\mathbf{y}=\mathbf{X}\boldsymbol{\beta} +\mathbf{z}$, where $\mathbf{X}$ is an $n\times p$ design matrix and $\mathbf{z}$ is an $n$-dimensional vector of independent Gaussian errors, each with variance $\sigma^{2}$. Our focus is on the recently introduced SLOPE estimator [Ann. Appl. Stat. 9 (2015) 1103–1140], which regularizes the least-squares estimates with the rank-dependent penalty $\sum_{1\le i\le p}\lambda_{i}|\widehat{\beta} |_{(i)}$, where $|\widehat{\beta} |_{(i)}$ is the $i$th largest magnitude of the fitted coefficients. Under Gaussian designs, where the entries of $\mathbf{X}$ are i.i.d. $\mathcal{N}(0,1/n)$, we show that SLOPE, with weights $\lambda_{i}$ just about equal to $\sigma\cdot\Phi^{-1}(1-iq/(2p))$ [$\Phi^{-1}(\alpha)$ is the $\alpha$th quantile of a standard normal and $q$ is a fixed number in $(0,1)$] achieves a squared error of estimation obeying \[\sup_{\|\boldsymbol{\beta} \|_{0}\le k}\mathbb{P} (\|\widehat{\boldsymbol {\beta}}_{\mathrm{SLOPE}}-\boldsymbol{\beta} \|^{2}>(1+\varepsilon) 2\sigma^{2}k\log(p/k))\longrightarrow 0\] as the dimension $p$ increases to $\infty$, and where $\varepsilon >0$ is an arbitrary small constant. This holds under a weak assumption on the $\ell_{0}$-sparsity level, namely, $k/p\rightarrow 0$ and $(k\log p)/n\rightarrow 0$, and is sharp in the sense that this is the best possible error any estimator can achieve. A remarkable feature is that SLOPE does not require any knowledge of the degree of sparsity, and yet automatically adapts to yield optimal total squared errors over a wide range of $\ell_{0}$-sparsity classes. We are not aware of any other estimator with this property.

Citation

Download Citation

Weijie Su. Emmanuel Candès. "SLOPE is adaptive to unknown sparsity and asymptotically minimax." Ann. Statist. 44 (3) 1038 - 1068, June 2016. https://doi.org/10.1214/15-AOS1397

Information

Received: 1 April 2015; Revised: 1 September 2015; Published: June 2016
First available in Project Euclid: 11 April 2016

zbMATH: 1338.62032
MathSciNet: MR3485953
Digital Object Identifier: 10.1214/15-AOS1397

Subjects:
Primary: 62C20
Secondary: 62G05, 62G10, 62J15

Rights: Copyright © 2016 Institute of Mathematical Statistics

JOURNAL ARTICLE
31 PAGES


SHARE
Vol.44 • No. 3 • June 2016
Back to Top