## The Annals of Applied Probability

- Ann. Appl. Probab.
- Volume 27, Number 4 (2017), 2098-2129.

### Phase transition in a sequential assignment problem on graphs

#### Abstract

We study the following sequential assignment problem on a finite graph $G=(V,E)$. Each edge $e\in E$ starts with an integer value $n_{e}\ge0$, and we write $n=\sum_{e\in E}n_{e}$. At time $t$, $1\le t\le n$, a uniformly random vertex $v\in V$ is generated, and one of the edges $f$ incident with $v$ must be selected. The value of $f$ is then decreased by $1$. There is a unit final reward if the configuration $(0,\ldots,0)$ is reached. Our main result is that there is a *phase transition*: as $n\to\infty$, the expected reward under the optimal policy approaches a constant $c_{G}>0$ when $(n_{e}/n:e\in E)$ converges to a point in the interior of a certain convex set $\mathcal{R}_{G}$, and goes to $0$ exponentially when $(n_{e}/n:e\in E)$ is bounded away from $\mathcal{R}_{G}$. We also obtain estimates in the near-critical region, that is when $(n_{e}/n:e\in E)$ lies close to $\partial\mathcal{R}_{G}$. We supply quantitative error bounds in our arguments.

#### Article information

**Source**

Ann. Appl. Probab., Volume 27, Number 4 (2017), 2098-2129.

**Dates**

Received: July 2015

Revised: September 2016

First available in Project Euclid: 30 August 2017

**Permanent link to this document**

https://projecteuclid.org/euclid.aoap/1504080027

**Digital Object Identifier**

doi:10.1214/16-AAP1250

**Mathematical Reviews number (MathSciNet)**

MR3693521

**Zentralblatt MATH identifier**

06803458

**Subjects**

Primary: 60K99: None of the above, but in this section

Secondary: 91A60: Probabilistic games; gambling [See also 60G40] 90C40: Markov and semi-Markov decision processes

**Keywords**

Phase transition critical phenomenon stochastic sequential assignment Markov decision process stochastic dynamic programming discrete stochastic optimal control

#### Citation

Járai, Antal A. Phase transition in a sequential assignment problem on graphs. Ann. Appl. Probab. 27 (2017), no. 4, 2098--2129. doi:10.1214/16-AAP1250. https://projecteuclid.org/euclid.aoap/1504080027