Lines of descent in the deterministic mutation–selection model with pairwise interaction

Abstract

We consider the mutation–selection differential equation with pairwise interaction (or, equivalently, the diploid mutation–selection equation) and establish the corresponding ancestral process, which is a random tree and a variant of the ancestral selection graph. The formal relation to the forward model is given via duality. To make the tree tractable, we prune branches upon mutations, thus reducing it to its informative parts. The hierarchies inherent in the tree are encoded systematically via tripod trees with weighted leaves; this leads to the stratified ancestral selection graph. The latter also satisfies a duality relation with the mutation–selection equation. Each of the dualities provides a stochastic representation of the solution of the differential equation. This allows us to connect the equilibria and their bifurcations to the long-term behaviour of the ancestral process. Furthermore, with the help of the stratified ancestral selection graph, we obtain explicit results about the ancestral type distribution in the case of unidirectional mutation.