The Annals of Applied Probability

Branching Brownian motion and selection in the spatial $\Lambda$-Fleming–Viot process

Alison Etheridge, Nic Freeman, Sarah Penington, and Daniel Straulino

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


We ask the question “when will natural selection on a gene in a spatially structured population cause a detectable trace in the patterns of genetic variation observed in the contemporary population?” We focus on the situation in which “neighbourhood size”, that is the effective local population density, is small. The genealogy relating individuals in a sample from the population is embedded in a spatial version of the ancestral selection graph and through applying a diffusive scaling to this object we show that whereas in dimensions at least three, selection is barely impeded by the spatial structure, in the most relevant dimension, $d=2$, selection must be stronger (by a factor of $\log(1/\mu)$ where $\mu$ is the neutral mutation rate) if we are to have a chance of detecting it. The case $d=1$ was handled in Etheridge, Freeman and Straulino (The Brownian net and selection in the spatial Lambda-Fleming–Viot. Preprint).

The mathematical interest is that although the system of branching and coalescing lineages that forms the ancestral selection graph converges to a branching Brownian motion, this reflects a delicate balance of a branching rate that grows to infinity and the instant annullation of almost all branches through coalescence caused by the strong local competition in the population.

Article information

Ann. Appl. Probab., Volume 27, Number 5 (2017), 2605-2645.

Received: December 2015
Revised: August 2016
First available in Project Euclid: 3 November 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60G99: None of the above, but in this section
Secondary: 92B05: General biology and biomathematics

Spatial Lambda–Fleming–Viot process branching coalescing natural selection branching Brownian motion population genetics


Etheridge, Alison; Freeman, Nic; Penington, Sarah; Straulino, Daniel. Branching Brownian motion and selection in the spatial $\Lambda$-Fleming–Viot process. Ann. Appl. Probab. 27 (2017), no. 5, 2605--2645. doi:10.1214/16-AAP1245.

Export citation


  • Barton, N. H. (1993). The probability of fixation of a favoured allele in a subdivided population. Genet. Res. 62 149–157.
  • Barton, N. H., Etheridge, A. M. and Véber, A. (2010). A new model for evolution in a spatial continuum. Electron. J. Probab. 15 162–216.
  • Barton, N. H., Etheridge, A. M., Kelleher, J. and Véber, A. (2013a). Genetic hitchhiking in spatially extended populations. Theor. Popul. Biol. 87 75–89.
  • Barton, N. H., Etheridge, A. M., Kelleher, J. and Véber, A. (2013b). Inference in two dimensions: Allele frequencies versus lengths of shared sequence blocks. Theor. Popul. Biol. 87 105–119.
  • Billingsley, P. (1995). Probability and Measure, 3rd ed. Wiley, New York.
  • Cherry, J. L. (2003). Selection in a subdivided population with local extinction and recolonization. Genetics 164 789–795.
  • Durrett, R. and Zähle, I. (2007). On the width of hybrid zones. Stochastic Process. Appl. 117 1751–1763.
  • Etheridge, A. M. (2008). Drift, draft and structure: Some mathematical models of evolution. Banach Center Publ. 80 121–144.
  • Etheridge, A., Freeman, N. and Straulino, D. (2015). The Brownian net and selection in the spatial Lambda-Fleming-Viot Process. Preprint. Available at arXiv:1506.01158.
  • Etheridge, A. M. and Kurtz, T. G. (2014). Genealogical constructions of population models. Preprint. Available at arXiv:1402.6724.
  • Etheridge, A. M. and Véber, A. (2012). The spatial $\Lambda$-Fleming-Viot process on a large torus: Genealogies in the presence of recombination. Ann. Appl. Probab. 22 2165–2209.
  • Etheridge, A. M., Véber, A. and Yu, F. (2014). Rescaling limits of the spatial Lambda-Fleming-Viot process with selection. Preprint. Available at arXiv:1406.5884.
  • Fisher, R. A. (1937). The wave of advance of advantageous genes. Ann. Eugenics 7 355–369.
  • Kallenberg, O. (2006). Foundations of Modern Probability. Springer, Berlin.
  • Kingman, J. F. C. (1993). Poisson Processes. Oxford Univ. Press, New York.
  • Krone, S. M. and Neuhauser, C. (1997). Ancestral processes with selection. Theor. Popul. Biol. 51 210–237.
  • Maruyama, T. (1970). On the fixation probability of mutant genes in a subdivided population. Genet. Res. 15 221–225.
  • Mueller, C., Mytnik, L. and Quastel, J. (2008). Small noise asymptotics of traveling waves. Markov Process. Related Fields 14 333–342.
  • Neuhauser, C. and Krone, S. M. (1997). Genealogies of samples in models with selection. Genetics 145 519–534.
  • Véber, A. and Wakolbinger, A. (2015). The spatial Lambda-Fleming–Viot process: An event-based construction and a lookdown representation. Ann. Inst. Henri Poincaré Probab. Stat. 51 570–598.
  • Whitlock, M. C. (2003). Fixation probability and time in subdivided populations. Genetics 164 767–779.