Evolving genealogies for branching populations under selection and competition

Abstract

For a continuous state branching process with two types of individuals which are subject to selection and density dependent competition, we characterize the joint evolution of population size, type configurations and genealogies as the unique strong solution of a system of SDEs. Our construction is achieved in the lookdown framework and provides a synthesis as well as a generalization of cases considered separately in two seminal papers by Donnelly and Kurtz (Ann. Appl. Probab. 9 (1999) 1091–1148; Ann. Probab. 27 (1999) 166–205), namely fluctuating population sizes under neutrality, and selection with constant population size. As a conceptual core in our approach we introduce the selective lookdown space which is obtained from its neutral counterpart through a state-dependent thinning of “potential” selection/competition events whose rates interact with the evolution of the type densities. The updates of the genealogical distance matrix at the “active” selection/competition events are obtained through an appropriate sampling from the selective lookdown space. The solution of the above mentioned system of SDEs is then mapped into the joint evolution of population size and symmetrized type configurations and genealogies, that is, marked distance matrix distributions. By means of Kurtz’ Markov mapping theorem, we characterize the latter process as the unique solution of a martingale problem. For the sake of transparency we restrict the main part of our presentation to a prototypical example with two types, which contains the essential features. In the final section we outline an extension to processes with multiple types including mutation.