Annals of Statistics

Bootstrap and permutation tests of independence for point processes

Mélisande Albert, Yann Bouret, Magalie Fromont, and Patricia Reynaud-Bouret

Full-text: Open access


Motivated by a neuroscience question about synchrony detection in spike train analysis, we deal with the independence testing problem for point processes. We introduce nonparametric test statistics, which are rescaled general $U$-statistics, whose corresponding critical values are constructed from bootstrap and randomization/permutation approaches, making as few assumptions as possible on the underlying distribution of the point processes. We derive general consistency results for the bootstrap and for the permutation w.r.t. Wasserstein’s metric, which induces weak convergence as well as convergence of second-order moments. The obtained bootstrap or permutation independence tests are thus proved to be asymptotically of the prescribed size, and to be consistent against any reasonable alternative. A simulation study is performed to illustrate the derived theoretical results, and to compare the performance of our new tests with existing ones in the neuroscientific literature.

Article information

Ann. Statist., Volume 43, Number 6 (2015), 2537-2564.

Received: July 2014
Revised: May 2015
First available in Project Euclid: 7 October 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62M07: Non-Markovian processes: hypothesis testing 62F40: Bootstrap, jackknife and other resampling methods 62E20: Asymptotic distribution theory 60G55: Point processes 60F05: Central limit and other weak theorems
Secondary: 62P10: Applications to biology and medical sciences

Independence test $U$-statistics point processes bootstrap randomization permutation neuroscience spike train analysis


Albert, Mélisande; Bouret, Yann; Fromont, Magalie; Reynaud-Bouret, Patricia. Bootstrap and permutation tests of independence for point processes. Ann. Statist. 43 (2015), no. 6, 2537--2564. doi:10.1214/15-AOS1351.

Export citation


  • [1] Albert, M., Bouret, Y., Fromont, M. and Reynaud-Bouret, P. (2015). A distribution free Unitary Events method based on delayed coincidence count. Available at arXiv:1505.06129.
  • [2] Albert, M., Bouret, Y., Fromont, M. and Reynaud-Bouret, P. (2015). Supplement to “Bootstrap and permutation tests of independence for point processes.” DOI:10.1214/15-AOS1351SUPP.
  • [3] Antoch, J. and Hušková, M. (2001). Permutation tests in change point analysis. Statist. Probab. Lett. 53 37–46.
  • [4] Arcones, M. A. and Giné, E. (1992). On the bootstrap of $U$ and $V$ statistics. Ann. Statist. 20 655–674.
  • [5] Arcones, M. A. and Giné, E. (1993). Limit theorems for $U$-processes. Ann. Probab. 21 1494–1542.
  • [6] Arlot, S., Blanchard, G. and Roquain, E. (2010). Some nonasymptotic results on resampling in high dimension. I. Confidence regions. Ann. Statist. 38 51–82.
  • [7] Arlot, S., Blanchard, G. and Roquain, E. (2010). Some nonasymptotic results on resampling in high dimension. II. Multiple tests. Ann. Statist. 38 83–99.
  • [8] Arvesen, J. N. (1969). Jackknifing $U$-statistics. Ann. Math. Statist. 40 2076–2100.
  • [9] Bickel, P. J. and Freedman, D. A. (1981). Some asymptotic theory for the bootstrap. Ann. Statist. 9 1196–1217.
  • [10] Billingsley, P. (2009). Convergence of Probability Measures. Wiley, New York.
  • [11] Blum, J. R., Kiefer, J. and Rosenblatt, M. (1961). Distribution free tests of independence based on the sample distribution function. Ann. Math. Statist. 32 485–498.
  • [12] Bouzebda, S. (2014). General tests of independence based on empirical processes indexed by functions. Stat. Methodol. 21 59–87.
  • [13] Brémaud, P. (1981). Point Processes and Queues: Martingale Dynamics. Springer, New York.
  • [14] Brémaud, P. and Massoulié, L. (1996). Stability of nonlinear Hawkes processes. Ann. Probab. 24 1563–1588.
  • [15] Bretagnolle, J. (1983). Lois limites du bootstrap de certaines fonctionnelles. Ann. Inst. H. Poincaré Sect. B (N.S.) 19 281–296.
  • [16] Chung, E. and Romano, J. P. (2013). Exact and asymptotically robust permutation tests. Ann. Statist. 41 484–507.
  • [17] Daley, D. J. and Vere-Jones, D. (2003). An Introduction to the Theory of Point Processes. Vol. I: Elementary Theory and Methods, 2nd ed. Springer, New York.
  • [18] Dehling, H. and Mikosch, T. (1994). Random quadratic forms and the bootstrap for $U$-statistics. J. Multivariate Anal. 51 392–413.
  • [19] Dudley, R. M. (2002). Real Analysis and Probability. Cambridge Studies in Advanced Mathematics 74. Cambridge Univ. Press, Cambridge.
  • [20] Durot, C. and Rozenholc, Y. (2006). An adaptive test for zero mean. Math. Methods Statist. 15 26–60.
  • [21] Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Ann. Statist. 7 1–26.
  • [22] Efron, B. and Tibshirani, R. J. (1993). An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability 57. Chapman & Hall, New York.
  • [23] Fisher, R. A. (1935). The Design of Experiments. Oliver and Boyd, Edinburgh.
  • [24] Fromont, M., Laurent, B., Lerasle, M. and Reynaud-Bouret, P. (2012). Kernels based tests with non-asymptotic bootstrap approaches for two-sample problems. Journal of Machine Learning Research: Workshop and Conference Proceedings, COLT 2012 23 23.1–23.22.
  • [25] Fromont, M., Laurent, B. and Reynaud-Bouret, P. (2013). The two-sample problem for Poisson processes: Adaptive tests with a nonasymptotic wild bootstrap approach. Ann. Statist. 41 1431–1461.
  • [26] Gerstein, G. L. and Perkel, D. H. (1969). Simultaneous recorded trains of action potentials: Analysis and functional interpretation. Science 164 828–830.
  • [27] Gretton, A. and Györfi, L. (2010). Consistent nonparametric tests of independence. J. Mach. Learn. Res. 11 1391–1423.
  • [28] Grün, S. (1996). Unitary joint-events in multiple-neuron spiking activity: Detection, significance and interpretation. Ph.D. thesis, Verlag Harri Deutsch, Thun.
  • [29] Grün, S., Diesmann, M. and Aertsen, A. M. (2010). Unitary events analysis. In Analysis of Parallel Spike Trains. Springer, Berlin.
  • [30] Grün, S., Diesmann, M., Grammont, F., Riehle, A. and Aertsen, A. M. (1999). Detecting unitary events without discretization of time. J. Neurosci. Methods 93 67–79.
  • [31] Hájek, J. (1961). Some extensions of the Wald–Wolfowitz–Noether theorem. Ann. Math. Statist. 32 506–523.
  • [32] Hansen, N. R., Reynaud-Bouret, P. and Rivoirard, V. (2015). Lasso and probabilistic inequalities for multivariate point processes. Bernoulli 21 83–143.
  • [33] Ho, S. T. and Chen, L. H. Y. (1978). An $L_{p}$ bound for the remainder in a combinatorial central limit theorem. Ann. Probab. 6 231–249.
  • [34] Hoeffding, W. (1948). A class of statistics with asymptotically normal distribution. Ann. Math. Statist. 19 293–325.
  • [35] Hoeffding, W. (1948). A non-parametric test of independence. Ann. Math. Statist. 19 546–557.
  • [36] Hoeffding, W. (1951). A combinatorial central limit theorem. Ann. Math. Statist. 22 558–566.
  • [37] Hoeffding, W. (1952). The large-sample power of tests based on permutations of observations. Ann. Math. Statist. 23 169–192.
  • [38] Horváth, L. and Hušková, M. (2005). Testing for changes using permutations of $U$-statistics. J. Statist. Plann. Inference 128 351–371.
  • [39] Hotelling, H. and Pabst, M. R. (1936). Rank correlation and tests of significance involving no assumption of normality. Ann. Math. Statist. 7 29–43.
  • [40] Hušková, M. and Janssen, P. (1993). Generalized bootstrap for studentized $U$-statistics: A rank statistic approach. Statist. Probab. Lett. 16 225–233.
  • [41] Janssen, A. and Pauls, T. (2003). How do bootstrap and permutation tests work? Ann. Statist. 31 768–806.
  • [42] Kendall, M. G. (1938). A new measure of rank correlation. Biometrika 30 81–93.
  • [43] Kirch, C. (2006). Resampling methods for the change analysis of dependent data. Ph.D. thesis, Universität zu Köln.
  • [44] Klebanov, L., Gordon, A., Xiao, Y., Land, H. and Yakovlev, A. (2006). A permutation test motivated by microarray data analysis. Comput. Statist. Data Anal. 50 3619–3628.
  • [45] Leucht, A. and Neumann, M. H. (2009). Consistency of general bootstrap methods for degenerate $U$-type and $V$-type statistics. J. Multivariate Anal. 100 1622–1633.
  • [46] Meintanis, S. G. and Iliopoulos, G. (2008). Fourier methods for testing multivariate independence. Comput. Statist. Data Anal. 52 1884–1895.
  • [47] Motoo, M. (1957). On the Hoeffding’s combinatorial central limit theorem. Ann. Inst. Statist. Math. Tokyo 8 145–154.
  • [48] Ogata, Y. (1981). On Lewis’ simulation method for point processes. IEEE Trans. Inform. Theory 27 23–31.
  • [49] Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 50 157–175.
  • [50] Pearson, K. (1911). On the probability that two independent distributions of frequency are really samples from the same population. Biometrika 8 250–254.
  • [51] Pernice, V., Staude, B., Cardanobile, S. and Rotter, S. (2011). How structure determines correlations in neuronal networks. PLoS Comput. Biol. 7 e1002059.
  • [52] Pesarin, F. and Salmaso, L. (2010). Permutation Tests for Complex Data: Theory, Applications and Software. Wiley, New York.
  • [53] Pipa, G., Diesmann, M. and Grün, S. (2003). Significance of joint-spike events based on trial-shuffling by efficient combinatorial methods. Complexity 8 79–86.
  • [54] Pipa, G. and Grün, S. (2003). Non-parametric significance estimation of joint-spike events by shuffling and resampling. Neurocomputing 52–54 31–37.
  • [55] Pitman, E. J. (1937). Significance tests which may be applied to samples from any populations. Suppl. J. R. Stat. Soc. 4 225–232.
  • [56] Pitman, E. J. (1938). Significance tests which may be applied to samples from any populations. The analysis of variance test. Biometrika 29 322–335.
  • [57] Pouzat, C. and Chaffiol, A. (2009). Automatic spike train analysis and report generation. An implementation with R, R2HTML and STAR. J. Neurosci. Methods 181 119–144.
  • [58] Reynaud-Bouret, P., Rivoirard, V., Grammont, F. and Tuleau-Malot, C. (2014). Goodness-of-fit tests and nonparametric adaptive estimation for spike train analysis. J. Math. Neurosci. 4 Art. 3, 41.
  • [59] Reynaud-Bouret, P., Rivoirard, V. and Tuleau-Malot, C. (2013). Inference of functional connectivity in neurosciences via Hawkes processes. In 1st IEEE Global Conference on Signal and Information Processing. Austin, Texas.
  • [60] Romano, J. P. (1987). Bootstrap and randomization tests of some nonparametric hypotheses. Technical Report 270, Dept. Statistics, Stanford Univ., Stanford, CA.
  • [61] Romano, J. P. (1988). A bootstrap revival of some nonparametric distance tests. J. Amer. Statist. Assoc. 83 698–708.
  • [62] Romano, J. P. (1989). Bootstrap and randomization tests of some nonparametric hypotheses. Ann. Statist. 17 141–159.
  • [63] Romano, J. P. and Wolf, M. (2005). Exact and approximate stepdown methods for multiple hypothesis testing. J. Amer. Statist. Assoc. 100 94–108.
  • [64] Rubin, H. and Vitale, R. A. (1980). Asymptotic distribution of symmetric statistics. Ann. Statist. 8 165–170.
  • [65] Sansonnet, L. and Tuleau-Malot, C. (2015). A model of Poissonian interactions and detection of dependence. Stat. Comput. 25 449–470.
  • [66] Scheffé, H. (1943). Statistical inference in the non-parametric case. Ann. Math. Statist. 14 305–332.
  • [67] Serfling, R. J. (2009). Approximation Theorems of Mathematical Statistics. Wiley, New York.
  • [68] Singer, W. (1993). Synchronization of cortical activity and its putative role in information processing and learning. Annu. Rev. Physiol. 55 349–374.
  • [69] Tuleau-Malot, C., Rouis, A., Grammont, F. and Reynaud-Bouret, P. (2014). Multiple tests based on a Gaussian approximation of the unitary events method with delayed coincidence count. Neural Comput. 26 1408–1454.
  • [70] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York.
  • [71] Varadarajan, V. S. (1958). On the convergence of sample probability distributions. Sankhyā 19 23–26.
  • [72] Wolfowitz, J. (1942). Additive partition functions and a class of statistical hypotheses. Ann. Math. Statist. 13 247–279.

Supplemental materials

  • Technical results and proofs of “Bootstrap and permutation tests of independence for point processes”. This Supplement consists of all the proofs. It also contains some additional results about nondegeneracy and the empirical centering assumption.