The Annals of Statistics
- Ann. Statist.
- Volume 35, Number 2 (2007), 870-887.
When do stepwise algorithms meet subset selection criteria?
Recent results in homotopy and solution paths demonstrate that certain well-designed greedy algorithms, with a range of values of the algorithmic parameter, can provide solution paths to a sequence of convex optimization problems. On the other hand, in regression many existing criteria in subset selection (including Cp, AIC, BIC, MDL, RIC, etc.) involve optimizing an objective function that contains a counting measure. The two optimization problems are formulated as (P1) and (P0) in the present paper. The latter is generally combinatoric and has been proven to be NP-hard. We study the conditions under which the two optimization problems have common solutions. Hence, in these situations a stepwise algorithm can be used to solve the seemingly unsolvable problem. Our main result is motivated by recent work in sparse representation, while two others emerge from different angles: a direct analysis of sufficiency and necessity and a condition on the mostly correlated covariates. An extreme example connected with least angle regression is of independent interest.
Ann. Statist., Volume 35, Number 2 (2007), 870-887.
First available in Project Euclid: 5 July 2007
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 62J07: Ridge regression; shrinkage estimators
Huo, Xiaoming; Ni, Xuelei (Sherry). When do stepwise algorithms meet subset selection criteria?. Ann. Statist. 35 (2007), no. 2, 870--887. doi:10.1214/009053606000001334. https://projecteuclid.org/euclid.aos/1183667297