Translator Disclaimer
December 2019 Sorted concave penalized regression
Long Feng, Cun-Hui Zhang
Ann. Statist. 47(6): 3069-3098 (December 2019). DOI: 10.1214/18-AOS1759

Abstract

The Lasso is biased. Concave penalized least squares estimation (PLSE) takes advantage of signal strength to reduce this bias, leading to sharper error bounds in prediction, coefficient estimation and variable selection. For prediction and estimation, the bias of the Lasso can be also reduced by taking a smaller penalty level than what selection consistency requires, but such smaller penalty level depends on the sparsity of the true coefficient vector. The sorted $\ell_{1}$ penalized estimation (Slope) was proposed for adaptation to such smaller penalty levels. However, the advantages of concave PLSE and Slope do not subsume each other. We propose sorted concave penalized estimation to combine the advantages of concave and sorted penalizations. We prove that sorted concave penalties adaptively choose the smaller penalty level and at the same time benefits from signal strength, especially when a significant proportion of signals are stronger than the corresponding adaptively selected penalty levels. A local convex approximation for sorted concave penalties, which extends the local linear and quadratic approximations for separable concave penalties, is developed to facilitate the computation of sorted concave PLSE and proven to possess desired prediction and estimation error bounds. Our analysis of prediction and estimation errors requires the restricted eigenvalue condition on the design, not beyond, and provides selection consistency under a required minimum signal strength condition in addition. Thus, our results also sharpens existing results on concave PLSE by removing the upper sparse eigenvalue component of the sparse Riesz condition.

Citation

Download Citation

Long Feng. Cun-Hui Zhang. "Sorted concave penalized regression." Ann. Statist. 47 (6) 3069 - 3098, December 2019. https://doi.org/10.1214/18-AOS1759

Information

Received: 1 November 2017; Revised: 1 June 2018; Published: December 2019
First available in Project Euclid: 31 October 2019

Digital Object Identifier: 10.1214/18-AOS1759

Subjects:
Primary: 62J05, 62J07
Secondary: 62H12

Rights: Copyright © 2019 Institute of Mathematical Statistics

JOURNAL ARTICLE
30 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

SHARE
Vol.47 • No. 6 • December 2019
Back to Top