## Advances in Applied Probability

- Adv. in Appl. Probab.
- Volume 46, Number 2 (2014), 446-477.

### Stochastic modeling of density-dependent diploid populations and the extinction vortex

#### Abstract

We model and study the genetic evolution and conservation of a population of diploid hermaphroditic organisms, evolving continuously in time and subject to resource competition. In the absence of mutations, the population follows a three-type, nonlinear birth-and-death process, in which birth rates are designed to integrate Mendelian reproduction. We are interested in the long-term genetic behavior of the population (adaptive dynamics), and in particular we compute the fixation probability of a slightly nonneutral allele in the absence of mutations, which involves finding the unique subpolynomial solution of a nonlinear three-dimensional recurrence relationship. This equation is simplified to a one-dimensional relationship which is proved to admit exactly one bounded solution. Adding rare mutations and rescaling time, we study the successive mutation fixations in the population, which are given by the jumps of a limiting Markov process on the genotypes space. At this time scale, we prove that the fixation rate of deleterious mutations increases with the number of already fixed mutations, which creates a vicious circle called the extinction vortex.

#### Article information

**Source**

Adv. in Appl. Probab., Volume 46, Number 2 (2014), 446-477.

**Dates**

First available in Project Euclid: 29 May 2014

**Permanent link to this document**

https://projecteuclid.org/euclid.aap/1401369702

**Digital Object Identifier**

doi:10.1239/aap/1401369702

**Mathematical Reviews number (MathSciNet)**

MR3215541

**Zentralblatt MATH identifier**

1325.92070

**Subjects**

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces) 60J80: Branching processes (Galton-Watson, birth-and-death, etc.)

Secondary: 60J27: Continuous-time Markov processes on discrete state spaces 92D25: Population dynamics (general) 92D15: Problems related to evolution

**Keywords**

Population genetics diploid population nonlinear birth-and-death process fixation probability Dirichlet problem multidimensional nonlinear recurrence equation extinction vortex

#### Citation

Coron, Camille. Stochastic modeling of density-dependent diploid populations and the extinction vortex. Adv. in Appl. Probab. 46 (2014), no. 2, 446--477. doi:10.1239/aap/1401369702. https://projecteuclid.org/euclid.aap/1401369702