The Annals of Statistics

High-dimensional change-point detection under sparse alternatives

Farida Enikeeva and Zaid Harchaoui

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

We consider the problem of detecting a change in mean in a sequence of high-dimensional Gaussian vectors. The change in mean may be occurring simultaneously in an unknown subset components. We propose a hypothesis test to detect the presence of a change-point and establish the detection boundary in different regimes under the assumption that the dimension tends to infinity and the length of the sequence grows with the dimension. A remarkable feature of the proposed test is that it does not require any knowledge of the subset of components in which the change in mean is occurring and yet automatically adapts to yield optimal rates of convergence over a wide range of statistical regimes.

Article information

Source
Ann. Statist., Volume 47, Number 4 (2019), 2051-2079.

Dates
Received: February 2014
Revised: March 2018
First available in Project Euclid: 21 May 2019

Permanent link to this document
https://projecteuclid.org/euclid.aos/1558425639

Digital Object Identifier
doi:10.1214/18-AOS1740

Mathematical Reviews number (MathSciNet)
MR3953444

Zentralblatt MATH identifier
07082279

Subjects
Primary: 62G10: Hypothesis testing 62H15: Hypothesis testing
Secondary: 60G35: Signal detection and filtering [See also 62M20, 93E10, 93E11, 94Axx]

Keywords
Change-point problem high-dimensional data minimax optimality sparsity

Citation

Enikeeva, Farida; Harchaoui, Zaid. High-dimensional change-point detection under sparse alternatives. Ann. Statist. 47 (2019), no. 4, 2051--2079. doi:10.1214/18-AOS1740. https://projecteuclid.org/euclid.aos/1558425639


Export citation

References

  • [1] Addario-Berry, L., Broutin, N., Devroye, L. and Lugosi, G. (2010). On combinatorial testing problems. Ann. Statist. 38 3063–3092.
  • [2] Bacon, R., Brinchmann, J., Richard, J., Contini, T., Drake, A., Franx, M., Tacchella, S., Vernet, J., Wisotzki, L., et al. (2015). The MUSE 3D view of the hubble deep field South. Astron. Astrophys. 575 A75.
  • [3] Baraud, Y. (2002). Non-asymptotic minimax rates of testing in signal detection. Bernoulli 8 577–606.
  • [4] Basseville, M. and Nikiforov, I. V. (1993). Detection of Abrupt Changes: Theory and Application. Prentice Hall, Englewood Cliffs, NJ.
  • [5] Boucheron, S., Lugosi, G. and Massart, P. (2013). Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford Univ. Press, Oxford.
  • [6] Bourguignon, S., Mary, D. and Slezak, E. (2011). Restoration of astrophysical spectra with sparsity constraints: Models and algorithms. IEEE J. Sel. Top. Signal Process. 5 1002–1013.
  • [7] Brodsky, B. E. and Darkhovsky, B. S. (1993). Nonparametric Methods in Change-Point Problems. Mathematics and Its Applications 243. Kluwer Academic, Dordrecht.
  • [8] Butucea, C. and Ingster, Y. I. (2013). Detection of a sparse submatrix of a high-dimensional noisy matrix. Bernoulli 19 2652–2688.
  • [9] Cai, T. T., Liu, W. and Xia, Y. (2014). Two-sample test of high dimensional means under dependence. J. R. Stat. Soc. Ser. B. Stat. Methodol. 76 349–372.
  • [10] Carlstein, E., Müller, H.-G. and Siegmund, D., eds. (1994). Change-Point Problems. Institute of Mathematical Statistics Lecture Notes—Monograph Series 23. IMS, Hayward, CA.
  • [11] Csörgő, M. and Horváth, L. (1997). Limit Theorems in Change-Point Analysis. Wiley, Chichester.
  • [12] Donoho, D. and Jin, J. (2004). Higher criticism for detecting sparse heterogeneous mixtures. Ann. Statist. 32 962–994.
  • [13] Enikeeva, F. and Harchaoui, Z. (2019). Supplement to “High-dimensional change-point detection under sparse alternatives.” DOI:10.1214/18-AOS1740SUPP.
  • [14] Harchaoui, Z., Vallet, F., Lung-Yut-Fong, A. and Cappé, O. (2009). A regularized kernel-based approach to unsupervised audio segmentation. In ICASSP 1665–1668.
  • [15] Hastie, T., Tibshirani, R. and Wainwright, M. (2015). Statistical Learning with Sparsity: The Lasso and Generalizations. Monographs on Statistics and Applied Probability 143. CRC Press, Boca Raton, FL.
  • [16] Ingster, Y. I., Pouet, C. and Tsybakov, A. B. (2009). Classification of sparse high-dimensional vectors. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 367 4427–4448.
  • [17] Ingster, Y. (1997). Some problems of hypothesis testing leading to infinitely divisible distributions. Math. Methods Statist. 6 47–69.
  • [18] Ingster, Y. and Suslina, I. A. (2002). On the detection of a signal with a known shape in a multichannel system. Zap. Nauchn. Sem. S.-Peterburg. Otdel. Mat. Inst. Steklov. (POMI) 294 88–112, 261.
  • [19] Ingster, Y. and Suslina, I. A. (2003). Nonparametric Goodness-of-Fit Testing Under Gaussian Models. Lecture Notes in Statistics 169. Springer, New York.
  • [20] Korostelev, A. and Korosteleva, O. (2011). Mathematical Statistics: Asymptotic Minimax Theory. Graduate Studies in Mathematics 119. Amer. Math. Soc., Providence, RI.
  • [21] Korostelev, A. and Lepski, O. (2008). On a multi-channel change-point problem. Math. Methods Statist. 17 187–197.
  • [22] Lehmann, E. L. and Romano, J. P. (2005). Testing Statistical Hypotheses, 3rd ed. Springer, New York.
  • [23] Lévy-Leduc, C. and Roueff, F. (2009). Detection and localization of change-points in high-dimensional network traffic data. Ann. Appl. Stat. 3 637–662.
  • [24] Meillier, C., Chatelain, F., Michel, O., Bacon, R., Piqueras, L., Bacher, R. and Ayasso, H. (2016). SELFI: An object-based, Bayesian method for faint emission line source detection in MUSE deep field data cubes. Astron. Astrophys. 588 A140.
  • [25] Olshen, A. B., Venkatraman, E. S., Lucito, R. and Wigler, M. (2004). Circular binary segmentation for the analysis of array-based DNA copy number data. Biostatistics 5 557–572.
  • [26] Paris, S., Mary, D. and Ferrari, A. (2013). Detection tests using sparse models, with application to hyperspectral data. IEEE Trans. Signal Process. 61 1481–1494.
  • [27] Shiryayev, A. N. (1978). Optimal Stopping Rules. Springer, New York–Heidelberg.
  • [28] Siegmund, D. (1985). Sequential Analysis: Tests and Confidence Intervals. Springer, New York.
  • [29] Stransky, N., Vallot, C., Reyal, F., Bernard-Pierrot, I., de Medina, S. G., Segraves, R., de Rycke, Y., Elvin, P., Cassidy, A., et al. (2006). Regional copy number-independent deregulation of transcription in cancer. Nat. Genet. 38 1386–1396.
  • [30] Vert, J. P. and Bleakley, K. (2010). Fast detection of multiple change-points shared by many signals using group LARS. In Advances in Neural Information Processing Systems 23 (J. D. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel and A. Culotta, eds.) 2343–2351. Curran Associates, Red Hook, NY.
  • [31] Xie, Y. and Siegmund, D. (2013). Sequential multi-sensor change-point detection. Ann. Statist. 41 670–692.
  • [32] Zhang, N. R., Siegmund, D. O., Ji, H. and Li, J. Z. (2010). Detecting simultaneous changepoints in multiple sequences. Biometrika 97 631–645.

Supplemental materials

  • Supplement to “High-dimensional change-point detection under sparse alternatives”. The supplementary material [13] contains omitted proofs and some additional simulation results.