Open Access
February 2008 Properties of higher criticism under strong dependence
Peter Hall, Jiashun Jin
Ann. Statist. 36(1): 381-402 (February 2008). DOI: 10.1214/009053607000000767

Abstract

The problem of signal detection using sparse, faint information is closely related to a variety of contemporary statistical problems, including the control of false-discovery rate, and classification using very high-dimensional data. Each problem can be solved by conducting a large number of simultaneous hypothesis tests, the properties of which are readily accessed under the assumption of independence. In this paper we address the case of dependent data, in the context of higher criticism methods for signal detection. Short-range dependence has no first-order impact on performance, but the situation changes dramatically under strong dependence. There, although higher criticism can continue to perform well, it can be bettered using methods based on differences of signal values or on the maximum of the data. The relatively inferior performance of higher criticism in such cases can be explained in terms of the fact that, under strong dependence, the higher criticism statistic behaves as though the data were partitioned into very large blocks, with all but a single representative of each block being eliminated from the dataset.

Citation

Download Citation

Peter Hall. Jiashun Jin. "Properties of higher criticism under strong dependence." Ann. Statist. 36 (1) 381 - 402, February 2008. https://doi.org/10.1214/009053607000000767

Information

Published: February 2008
First available in Project Euclid: 1 February 2008

zbMATH: 1139.62049
MathSciNet: MR2387976
Digital Object Identifier: 10.1214/009053607000000767

Subjects:
Primary: 62G10 , 62M10
Secondary: 62G20 , 62G32

Keywords: Correlation , dependent data , faint information , Gaussian process , signal detection , simultaneous hypothesis testing , Sparsity

Rights: Copyright © 2008 Institute of Mathematical Statistics

Vol.36 • No. 1 • February 2008
Back to Top