On the exact Berk-Jones statistics and their $p$-value calculation

Abstract

Continuous goodness-of-fit testing is a classical problem in statistics. Despite having low power for detecting deviations at the tail of a distribution, the most popular test is based on the Kolmogorov-Smirnov statistic. While similar variance-weighted statistics such as Anderson-Darling and the Higher Criticism statistic give more weight to tail deviations, as shown in various works, they still mishandle the extreme tails.

As a viable alternative, in this paper we study some of the statistical properties of the exact $M_{n}$ statistics of Berk and Jones. In particular we show that they are consistent and asymptotically optimal for detecting a wide range of rare-weak mixture models. Additionally, we present a new computationally efficient method to calculate $p$-values for any supremum-based one-sided statistic, including the one-sided $M_{n}^{+},M_{n}^{-}$ and $R_{n}^{+},R_{n}^{-}$ statistics of Berk and Jones and the Higher Criticism statistic. Finally, we show that $M_{n}$ compares favorably to related statistics in several finite-sample simulations.