## Annals of Statistics

### Negative association, ordering and convergence of resampling methods

#### Abstract

We study convergence and convergence rates for resampling schemes. Our first main result is a general consistency theorem based on the notion of negative association, which is applied to establish the almost sure weak convergence of measures output from Kitagawa’s [J. Comput. Graph. Statist. 5 (1996) 1–25] stratified resampling method. Carpenter, Ckiffird and Fearnhead’s [IEE Proc. Radar Sonar Navig. 146 (1999) 2–7] systematic resampling method is similar in structure but can fail to converge depending on the order of the input samples. We introduce a new resampling algorithm based on a stochastic rounding technique of [In 42nd IEEE Symposium on Foundations of Computer Science (Las Vegas, NV, 2001) (2001) 588–597 IEEE Computer Soc.], which shares some attractive properties of systematic resampling, but which exhibits negative association and, therefore, converges irrespective of the order of the input samples. We confirm a conjecture made by [J. Comput. Graph. Statist. 5 (1996) 1–25] that ordering input samples by their states in $\mathbb{R}$ yields a faster rate of convergence; we establish that when particles are ordered using the Hilbert curve in $\mathbb{R}^{d}$, the variance of the resampling error is ${\scriptstyle\mathcal{O}}(N^{-(1+1/d)})$ under mild conditions, where $N$ is the number of particles. We use these results to establish asymptotic properties of particle algorithms based on resampling schemes that differ from multinomial resampling.

#### Article information

Source
Ann. Statist., Volume 47, Number 4 (2019), 2236-2260.

Dates
Revised: June 2018
First available in Project Euclid: 21 May 2019

https://projecteuclid.org/euclid.aos/1558425644

Digital Object Identifier
doi:10.1214/18-AOS1746

Mathematical Reviews number (MathSciNet)
MR3953450

Zentralblatt MATH identifier
07082285

#### Citation

Gerber, Mathieu; Chopin, Nicolas; Whiteley, Nick. Negative association, ordering and convergence of resampling methods. Ann. Statist. 47 (2019), no. 4, 2236--2260. doi:10.1214/18-AOS1746. https://projecteuclid.org/euclid.aos/1558425644

#### References

• Aistleitner, C. and Dick, J. (2015). Functions of bounded variation, signed measures, and a general Koksma–Hlawka inequality. Acta Arith. 167 143–171.
• Andrieu, C., Doucet, A. and Holenstein, R. (2010). Particle Markov chain Monte Carlo methods. J. R. Stat. Soc. Ser. B. Stat. Methodol. 72 269–342.
• Barbe, P. and Bertail, P. (1995). The Weighted Bootstrap. Lecture Notes in Statistics 98. Springer, New York.
• Bertail, P., Chautru, E. and Clémençon, S. (2017). Empirical processes in survey sampling with (conditional) Poisson designs. Scand. J. Stat. 44 97–111.
• Bickel, P. J. and Millar, P. W. (1992). Uniform convergence of probability measures on classes of functions. Statist. Sinica 2 1–15.
• Brown Kramer, J., Cutler, J. and Radcliffe, A. J. (2011). Negative dependence and Srinivasan’s sampling process. Combin. Probab. Comput. 20 347–361.
• Cappé, O., Moulines, E. and Rydén, T. (2005). Inference in Hidden Markov Models. Springer, New York.
• Carpenter, J., Clifford, P. and Fearnhead, P. (1999). Improved particle filter for nonlinear problems. IEE Proc. Radar Sonar Navig. 146 2–7.
• Chopin, N. (2004). Central limit theorem for sequential Monte Carlo methods and its application to Bayesian inference. Ann. Statist. 32 2385–2411.
• Cornebise, J., Moulines, É. and Olsson, J. (2008). Adaptive methods for sequential importance sampling with application to state space models. Stat. Comput. 18 461–480.
• Crisan, D. (2001). Particle filters—A theoretical perspective. In Sequential Monte Carlo Methods in Practice (A. Doucet, N. de Freitas and N. J. Gordon, eds.) 17–41. Springer, New York.
• Crisan, D. and Doucet, A. (2002). A survey of convergence results on particle filtering methods for practitioners. IEEE Trans. Signal Process. 50 736–746.
• Del Moral, P. and Doucet, A. (2003). On a class of genealogical and interacting Metropolis models. In Séminaire de Probabilités XXXVII. Lecture Notes in Math. 1832 415–446. Springer, Berlin.
• Deville, J.-C. and Tillé, Y. (1998). Unequal probability sampling without replacement through a splitting method. Biometrika 85 89–101.
• Douc, R., Cappé, O. and Moulines, E. (2005). Comparison of resampling schemes for particle filtering. In ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis 64–69. IEEE, New York.
• Douc, R., Moulines, É. and Olsson, J. (2009). Optimality of the auxiliary particle filter. Probab. Math. Statist. 29 1–28.
• Doucet, A., de Freitas, N. and Gordon, N. J. (2001). Sequential Monte Carlo Methods in Practice. Springer, New York.
• Dubhashi, D., Jonasson, J. and Ranjan, D. (2007). Positive influence and negative dependence. Combin. Probab. Comput. 16 29–41.
• Dubhashi, D. and Ranjan, D. (1998). Balls and bins: A study in negative dependence. Random Structures Algorithms 13 99–124.
• Gerber, M. and Chopin, N. (2015). Sequential quasi Monte Carlo. J. R. Stat. Soc. Ser. B. Stat. Methodol. 77 509–579.
• Gerber, M. and Chopin, N. (2017). Convergence of sequential quasi-Monte Carlo smoothing algorithms. Bernoulli 23 2951–2987.
• Gerber, M., Chopin, N. and Whiteley, N. (2019). Supplement to “Negative association, ordering and convergence of resampling methods.” DOI:10.1214/18-AOS1746SUPP.
• Gordon, N. J., Salmond, D. J. and Smith, A. F. M. (1993). Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F, Commun. Radar Signal Process. 140 107–113.
• Guarniero, P., Johansen, A. M. and Lee, A. (2017). The iterated auxiliary particle filter. J. Amer. Statist. Assoc. 112 1636–1647.
• He, Z. and Owen, A. B. (2016). Extensible grids: Uniform sampling on a space filling curve. J. R. Stat. Soc. Ser. B. Stat. Methodol. 78 917–931.
• Jaffard, S. and Nicolay, S. (2009). Pointwise smoothness of space-filling functions. Appl. Comput. Harmon. Anal. 26 181–199.
• Joag-Dev, K. and Proschan, F. (1983). Negative association of random variables, with applications. Ann. Statist. 11 286–295.
• Johansen, A. M. and Doucet, A. (2008). A note on auxiliary particle filters. Statist. Probab. Lett. 78 1498–1504.
• Kitagawa, G. (1996). Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. J. Comput. Graph. Statist. 5 1–25.
• Kuipers, L. and Niederreiter, H. (1974). Uniform Distribution of Sequences: Pure and Applied Mathematics. Wiley-Interscience [John Wiley & Sons], New York–London–Sydney.
• Künsch, H. R. (2005). Recursive Monte Carlo filters: Algorithms and theoretical analysis. Ann. Statist. 33 1983–2021.
• L’Ecuyer, P. and Lemieux, C. (2000). Variance reduction via lattice rules. Manage. Sci. 46 1214–1235.
• Liu, J. S. and Chen, R. (1998). Sequential Monte Carlo methods for dynamic systems. J. Amer. Statist. Assoc. 93 1032–1044.
• Madow, W. G. and Madow, L. H. (1944). On the theory of systematic sampling. I. Ann. Math. Stat. 15 1–24.
• Niederreiter, H. (1992). Random Number Generation and Quasi-Monte Carlo Methods. CBMS-NSF Regional Conference Series in Applied Mathematics 63. SIAM, Philadelphia, PA.
• Pitt, M. K. and Shephard, N. (1999). Filtering via simulation: Auxiliary particle filters. J. Amer. Statist. Assoc. 94 590–599.
• Pólya, G. (1920). Über den zentralen Grenzwertsatz der Wahrscheinlichkeitsrechnung und das Momentenproblem. Math. Z. 8 171–181.
• Schretter, C., He, Z., Gerber, M., Chopin, N. and Niederreiter, H. (2016). Van der Corput and golden ratio sequences along the Hilbert space-filling curve. In Monte Carlo and Quasi-Monte Carlo Methods. Springer Proc. Math. Stat. 163 531–544. Springer, Cham.
• Shao, Q.-M. (2000). A comparison theorem on moment inequalities between negatively associated and independent random variables. J. Theoret. Probab. 13 343–356.
• Srinivasan, A. (2001). Distributions on level-sets with applications to approximation algorithms. In 42nd IEEE Symposium on Foundations of Computer Science (Las Vegas, NV, 2001) 588–597. IEEE Computer Soc., Los Alamitos, CA.
• Stein, M. (1987). Large sample properties of simulations using Latin hypercube sampling. Technometrics 29 143–151.
• Tillé, Y. (2006). Sampling Algorithms. Springer, New York.
• van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics 3. Cambridge Univ. Press, Cambridge.
• Zumbusch, G. (2003). Parallel Multilevel Methods. Springer, Berlin.

#### Supplemental materials

• Supplementary material: Proofs. This supplement contains all the proofs.