Electronic Journal of Probability

Trees within trees: simple nested coalescents

Airam Blancas, Jean-Jil Duchamps, Amaury Lambert, and Arno Siri-Jégousse

Full-text: Open access


We consider the compact space of pairs of nested partitions of $\mathbb{N} $, where by analogy with models used in molecular evolution, we call “gene partition” the finer partition and “species partition” the coarser one. We introduce the class of nondecreasing processes valued in nested partitions, assumed Markovian and with exchangeable semigroup. These processes are said simple when each partition only undergoes one coalescence event at a time (but possibly the same time). Simple nested exchangeable coalescent (SNEC) processes can be seen as the extension of $\Lambda $-coalescents to nested partitions. We characterize the law of SNEC processes as follows. In the absence of gene coalescences, species blocks undergo $\Lambda $-coalescent type events and in the absence of species coalescences, gene blocks lying in the same species block undergo i.i.d. $\Lambda $-coalescents. Simultaneous coalescence of the gene and species partitions are governed by an intensity measure $\nu _s$ on $(0,1]\times{\mathcal M} _1 ([0,1])$ providing the frequency of species merging and the law in which are drawn (independently) the frequencies of genes merging in each coalescing species block. As an application, we also study the conditions under which a SNEC process comes down from infinity.

Article information

Electron. J. Probab., Volume 23 (2018), paper no. 94, 27 pp.

Received: 6 March 2018
Accepted: 3 September 2018
First available in Project Euclid: 18 September 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60G09: Exchangeability 60G57: Random measures 60J35: Transition functions, generators and resolvents [See also 47D03, 47D07] 60J75: Jump processes 92D10: Genetics {For genetic algebras, see 17D92} 92D15: Problems related to evolution

lambda-coalescent exchangeable partition coming down from infinity random tree gene tree population genetics species tree phylogenetics evolution

Creative Commons Attribution 4.0 International License.


Blancas, Airam; Duchamps, Jean-Jil; Lambert, Amaury; Siri-Jégousse, Arno. Trees within trees: simple nested coalescents. Electron. J. Probab. 23 (2018), paper no. 94, 27 pp. doi:10.1214/18-EJP219. https://projecteuclid.org/euclid.ejp/1537257886

Export citation


  • [1] Julien Berestycki, Nathanaël Berestycki, and Vlada Limic, A small-time coupling between $\Lambda $-coalescents and branching processes, The Annals of Applied Probability 24 (2014), no. 2, 449–475 (en).
  • [2] Julien Berestycki, Nathanaël Berestycki, and Jason Schweinsberg, The genealogy of branching Brownian motion with absorption, The Annals of Probability 41 (2013), no. 2, 527–618 (en).
  • [3] Nathanaël Berestycki, Recent progress in coalescent theory, Ensaios Matemáticos 16 (2009), no. 1, 1–193.
  • [4] Jean Bertoin, Random fragmentation and coagulation processes, Cambridge University Press, Cambridge, 2006.
  • [5] Jean Bertoin and Jean-François Le Gall, Stochastic flows associated to coalescent processes, Probability Theory and Related Fields 126 (2003), no. 2, 261–288.
  • [6] Jean Bertoin and Jean-François Le Gall, Stochastic flows associated to coalescent processes. III. Limit theorems, Illinois Journal of Mathematics 50 (2006), no. 1-4, 147–181.
  • [7] Airam Blancas, Tim Rogers, Jason Schweinsberg, and Arno Siri-Jégousse, The nested Kingman coalescent: Speed of coming down from infinity, arXiv:1803.08973 [math] (2018).
  • [8] Airam Blancas and Anton Wakolbinger, A representation for the semigroup of a two-level Fleming-Viot process in terms of the Kingman nested coalescent, In preparation.
  • [9] Erwin Bolthausen and Alain-Sol Sznitman, On Ruelle’s probability cascades and an abstract cavity method, Communications in Mathematical Physics 197 (1998), no. 2, 247–276.
  • [10] Éric Brunet and Bernard Derrida, Genealogies in simple models of evolution, Journal of Statistical Mechanics: Theory and Experiment 2013 (2013), no. 01, P01006.
  • [11] Donald A. Dawson, Multilevel mutation-selection systems and set-valued duals, Journal of Mathematical Biology 76 (2018), no. 1-2, 295–378 (en).
  • [12] James H Degnan and Noah A Rosenberg, Gene tree discordance, phylogenetic inference and the multispecies coalescent, Trends in ecology & evolution 24 (2009), no. 6, 332–340.
  • [13] Michael M Desai, Aleksandra M Walczak, and Daniel S Fisher, Genetic diversity and the structure of genealogies in rapidly adapting populations, Genetics 193 (2013), no. 2, 565–585.
  • [14] Jeff J Doyle, Trees within trees: genes and species, molecules and morphology, Systematic Biology 46 (1997), no. 3, 537–553.
  • [15] Jean-Jil Duchamps, Trees within trees II: Nested Fragmentations, arXiv:1807.05951 (2018).
  • [16] Rick Durrett and Jason Schweinsberg, A coalescent model for the effect of advantageous mutations on the genealogy of a population, Stochastic processes and their applications 115 (2005), no. 10, 1628–1657.
  • [17] Bjarki Eldon and John Wakeley, Coalescent processes when the distribution of offspring number among individuals is highly skewed, Genetics 172 (2006), no. 4, 2621–2633.
  • [18] Alison Etheridge, Some mathematical models from population genetics: École d’été de probabilités de Saint-Flour XXXIX-2009, Lecture notes in mathematics, Springer, 2011.
  • [19] Joseph Felsenstein, Inferring phylogenies, vol. 2, Sinauer associates Sunderland, MA, 2004.
  • [20] Félix Foutel-Rodier, Amaury Lambert, and Emmanuel Schertzer, Exchangeable coalescents, ultrametric spaces, nested interval-partitions: a unifying approach, arXiv:1807.05165 (2018).
  • [21] Bryan T Grenfell, Oliver G Pybus, Julia R Gog, James LN Wood, Janet M Daly, Jenny A Mumford, and Edward C Holmes, Unifying the epidemiological and evolutionary dynamics of pathogens, Science 303 (2004), no. 5656, 327–332.
  • [22] Joseph Heled and Alexei J Drummond, Bayesian inference of species trees from multilocus data, Molecular biology and evolution 27 (2009), no. 3, 570–580.
  • [23] Olav Kallenberg, Probabilistic symmetries and invariance principles, Probability and Its Applications, Springer-Verlag, New York, 2005 (en).
  • [24] J.F.C. Kingman, The coalescent, Stochastic processes and their applications 13 (1982), no. 3, 235–248.
  • [25] Amaury Lambert, Population dynamics and random genealogies, Stochastic Models 24 (2008), no. sup1, 45–163.
  • [26] Amaury Lambert, Probabilistic models for the (sub)tree(s) of life, Braz. J. Probab. Stat. 31 (2017), no. 3, 415–475.
  • [27] Amaury Lambert, Random ultrametric trees and applications, ESAIM: Procs 60 (2017), 70–89.
  • [28] Amaury Lambert and Emmanuel Schertzer, Coagulation-transport equations and the nested coalescents, arXiv:1807.09153 (2018).
  • [29] Wayne P Maddison, Gene trees in species trees, Systematic biology 46 (1997), no. 3, 523–536.
  • [30] Sebastian Matuszewski, Marcel E Hildebrandt, Guillaume Achaz, and Jeffrey D Jensen, Coalescent processes with skewed offspring distributions and non-equilibrium demography, Genetics (2017), genetics–300499.
  • [31] Richard A. Neher and Oskar Hallatschek, Genealogies of rapidly adapting populations, Proceedings of the National Academy of Sciences 110 (2013), no. 2, 437–442.
  • [32] Masatoshi Nei and Sudhir Kumar, Molecular evolution and phylogenetics, Oxford university press, 2000.
  • [33] Roderic DM Page and Michael A Charleston, From gene to organismal phylogeny: reconciled trees and the gene tree/species tree problem, Molecular phylogenetics and evolution 7 (1997), no. 2, 231–240.
  • [34] Roderic DM Page and Michael A Charleston, Trees within trees: phylogeny and historical associations, Trends in Ecology & Evolution 13 (1998), no. 9, 356–359.
  • [35] Jim Pitman, Coalescents with multiple collisions, The Annals of Probability 27 (1999), no. 4, 1870–1902 (en).
  • [36] Noah A Rosenberg, The probability of topological concordance of gene trees and species trees, Theoretical population biology 61 (2002), no. 2, 225–247.
  • [37] Serik Sagitov, The general coalescent with asynchronous mergers of ancestral lines, Journal of Applied Probability 36 (1999), no. 4, 1116–1125.
  • [38] Jason Schweinsberg, Coalescents with simultaneous multiple collisions, Electronic Journal of Probability 5 (2000) (EN).
  • [39] Jason Schweinsberg, A necessary and sufficient condition for the $\Lambda $-coalescent to come down from infinity., Electronic Communications in Probability 5 (2000), 1–11 (EN).
  • [40] Jason Schweinsberg, Coalescent processes obtained from supercritical Galton-Watson processes, Stochastic Processes and their Applications 106 (2003), no. 1, 107–139.
  • [41] Jason Schweinsberg, Rigorous results for a population model with selection II: genealogy of the population, Electronic Journal of Probability 22 (2017).
  • [42] Charles Semple and Mike A Steel, Phylogenetics, vol. 24, Oxford University Press, 2003.
  • [43] Gergely J Szöllősi, Eric Tannier, Vincent Daubin, and Bastien Boussau, The inference of gene trees with species trees, Systematic biology 64 (2014), no. 1, e42–e62.
  • [44] Aurelien Tellier and Christophe Lemaire, Coalescence 2.0: A multiple branching of recent theoretical developments and their applications, Molecular ecology 23 (2014), no. 11, 2637–2652.
  • [45] Erik M Volz, Katia Koelle, and Trevor Bedford, Viral phylodynamics, PLoS Computational Biology 9 (2013), no. 3, e1002947.