## The Annals of Statistics

### Two-sample Kolmogorov–Smirnov-type tests revisited: Old and new tests in terms of local levels

#### Abstract

From a multiple testing viewpoint, Kolmogorov–Smirnov (KS)-type tests are union-intersection tests which can be redefined in terms of local levels. The local level perspective offers a new viewpoint on ranges of sensitivity of KS-type tests and the design of new tests. We study the finite and asymptotic local level behavior of weighted KS tests which are either tail, intermediate or central sensitive. Furthermore, we provide new tests with approximately equal local levels and prove that the asymptotics of such tests with sample sizes $m$ and $n$ coincides with the asymptotics of one-sample higher criticism tests with sample size $\min (m,n)$. We compare the overall power of various tests and introduce local powers that are in line with local levels. Finally, suitably parameterized local level shape functions can be used to design new tests. We illustrate how to combine tests with different sensitivity in terms of local levels.

#### Article information

Source
Ann. Statist., Volume 46, Number 6A (2018), 3014-3037.

Dates
Revised: September 2017
First available in Project Euclid: 7 September 2018

https://projecteuclid.org/euclid.aos/1536307241

Digital Object Identifier
doi:10.1214/17-AOS1647

Mathematical Reviews number (MathSciNet)
MR3851763

#### Citation

Finner, Helmut; Gontscharuk, Veronika. Two-sample Kolmogorov–Smirnov-type tests revisited: Old and new tests in terms of local levels. Ann. Statist. 46 (2018), no. 6A, 3014--3037. doi:10.1214/17-AOS1647. https://projecteuclid.org/euclid.aos/1536307241

#### References

• [1] Aldor-Noiman, S., Brown, L. D., Buja, A., Rolke, W. and Stine, R. A. (2013). The power to see: A new graphical test of normality. Amer. Statist. 67 249–260.
• [2] Aldor-Noiman, S., Brown, L. D., Buja, A., Rolke, W. and Stine, R. A. (2014). Correction to: “The power to see: A new graphical test of normality.” [Amer. Statist. 67(4) (2013) 249–260. MR3303820] Amer. Statist. 68 318.
• [3] Barnard, G. A. (1947). Significance test for $2\times 2$ tables. Biometrika 34 123–138.
• [4] Berk, R. H. and Jones, D. H. (1978). Relatively optimal combinations of test statistics. Scand. J. Stat. 5 158–162.
• [5] Berk, R. H. and Jones, D. H. (1979). Goodness-of-fit test statistics that dominate the Kolmogorov statistics. Z. Wahrsch. Verw. Gebiete 47 47–59.
• [6] Canner, P. L. (1975). A simulation study of one- and two-sample Kolmogorov–Smirnov statistics with a particular weight function. J. Amer. Statist. Assoc. 70 209–211.
• [7] Csörgő, M., Csörgő, S., Horváth, L. and Mason, D. M. (1986). Weighted empirical and quantile processes. Ann. Probab. 14 31–85.
• [8] Doksum, K. A. and Sievers, G. L. (1976). Plotting with confidence: Graphical comparisons of two populations. Biometrika 63 421–434.
• [9] Donoho, D. and Jin, J. (2004). Higher criticism for detecting sparse heterogeneous mixtures. Ann. Statist. 32 962–994.
• [10] Donoho, D. and Jin, J. (2015). Higher criticism for large-scale inference, especially for rare and weak effects. Statist. Sci. 30 1–25.
• [11] Finner, H. and Gontscharuk, V. (2018). Supplement A to “Two-sample Kolmogorov-Smirnov type tests revisited: Old and new tests in terms of local levels”: Proofs and computation of global levels. DOI:10.1214/17-AOS1647SUPPA.
• [12] Finner, H. and Gontscharuk, V. (2018). Supplement B to “Two-sample Kolmogorov-Smirnov type tests revisited: Old and new tests in terms of local levels”: Animated graphics of local levels. DOI:10.1214/17-AOS1647SUPPB.
• [13] Finner, H. and Strassburger, K. (2002). Structural properties of UMPU-tests for 2 $\times$ 2-tables and some applications. J. Statist. Plann. Inference 104 103–120.
• [14] Frey, J. (2008). Optimal distribution-free confidence bands for a distribution function. J. Statist. Plann. Inference 138 3086–3098.
• [15] Gontscharuk, V. and Finner, H. (2017). Asymptotics of goodness-of-fit tests based on minimum $p$-value statistics. Comm. Statist. Theory Methods 46 2332–2342.
• [16] Gontscharuk, V., Landwehr, S. and Finner, H. (2015). The intermediates take it all: Asymptotics of higher criticism statistics and a powerful alternative based on equal local levels. Biom. J. 57 159–180.
• [17] Gontscharuk, V., Landwehr, S. and Finner, H. (2016). Goodness of fit tests in terms of local levels with special emphasis on higher criticism tests. Bernoulli 22 1331–1363.
• [18] Hodges, J. L. (1958). The significance probability of the Smirnov two-sample test. Ark. Mat. 3 469–486.
• [19] Jager, L. and Wellner, J. A. (2004). On the “Poisson boundaries” of the family of weighted Kolmogorov statistics. In A Festschrift for Herman Rubin. Institute of Mathematical Statistics Lecture Notes—Monograph Series 45 319–331. IMS, Beachwood, OH.
• [20] Janssen, A. (2000). Global power functions of goodness of fit tests. Ann. Statist. 28 239–253.
• [21] Kolmogorov, A. (1933). Sulla determinazione empirica di una legge di distribuzione. G. Inst. Ital. Attuari 4 83–91. Translated by Q. Meneghini as On the empirical determination of a distribution function. In Breakthroughs in Statistics II. Springer Series in Statistics (Perspectives in Statistics) (S. Kotz and N. L. Johnson, eds.) 106–113. Springer, New York.
• [22] Mason, D. M. (1983). The asymptotic distribution of weighted empirical distribution functions. Stochastic Process. Appl. 15 99–109.
• [23] Mason, D. M. and Schuenemeyer, J. H. (1983). A modified Kolmogorov–Smirnov test sensitive to tail alternatives. Ann. Statist. 11 933–946.
• [24] Mason, D. M. and Schuenemeyer, J. H. (1992). Correction to: “A modified Kolmogorov–Smirnov test sensitive to tail alternatives.” [Ann. Statist. 11(3) (1983) 933–946. MR0707943] Ann. Statist. 20 620–621.
• [25] Pyke, R. (1959). The supremum and infimum of the Poisson process. Ann. Math. Stat. 30 568–576.
• [26] Smirnoff, N. (1939). Sur les écarts de la courbe de distribution empirique. Rec. Math. N.S. [Mat. Sbornik] 6(48) 3–26.
• [27] Smirnov, N. (1939). On the estimation of the discrepancy between empirical curves of distribution for two independent samples. Moscow Univ. Math. Bull. 2 3–16.
• [28] Steck, G. P. (1969). The Smirnov two sample tests as rank tests. Ann. Math. Stat. 40 1449–1466.
• [29] Xu, X., Ding, X. and Zhao, S. (2009). The reduction of the average width of confidence bands for an unknown continuous distribution function. J. Stat. Comput. Simul. 79 335–347.

#### Supplemental materials

• Supplement A: Proofs and computation of global levels. In Section A1, we prove Lemma 3.1. Section A2 focuses on the computation of global levels. Proofs of asymptotic results in Sections 3.2 and 4.2 are given in Section A3. Section A4 provides technical results for proofs in Section A3.
• Supplement B: Animated graphics of local levels. In this supplement, we illustrate the convergence of local levels related to weighted KS as well as minP tests to the corresponding asymptotic counterparts by means of animated graphics.