Open Access
June 2008 A study of pre-validation
Holger Höfling, Robert Tibshirani
Ann. Appl. Stat. 2(2): 643-664 (June 2008). DOI: 10.1214/07-AOAS152

Abstract

Given a predictor of outcome derived from a high-dimensional dataset, pre-validation is a useful technique for comparing it to competing predictors on the same dataset. For microarray data, it allows one to compare a newly derived predictor for disease outcome to standard clinical predictors on the same dataset. We study pre-validation analytically to determine if the inferences drawn from it are valid. We show that while pre-validation generally works well, the straightforward “one degree of freedom” analytical test from pre-validation can be biased and we propose a permutation test to remedy this problem. In simulation studies, we show that the permutation test has the nominal level and achieves roughly the same power as the analytical test.

Citation

Download Citation

Holger Höfling. Robert Tibshirani. "A study of pre-validation." Ann. Appl. Stat. 2 (2) 643 - 664, June 2008. https://doi.org/10.1214/07-AOAS152

Information

Published: June 2008
First available in Project Euclid: 3 July 2008

zbMATH: 1273.62126
MathSciNet: MR2524350
Digital Object Identifier: 10.1214/07-AOAS152

Keywords: cross-validation , Hypothesis testing , inference , microarray , point estimation

Rights: Copyright © 2008 Institute of Mathematical Statistics

Vol.2 • No. 2 • June 2008
Back to Top