## The Annals of Applied Probability

### Phase transition in a sequential assignment problem on graphs

Antal A. Járai

#### Abstract

We study the following sequential assignment problem on a finite graph $G=(V,E)$. Each edge $e\in E$ starts with an integer value $n_{e}\ge0$, and we write $n=\sum_{e\in E}n_{e}$. At time $t$, $1\le t\le n$, a uniformly random vertex $v\in V$ is generated, and one of the edges $f$ incident with $v$ must be selected. The value of $f$ is then decreased by $1$. There is a unit final reward if the configuration $(0,\ldots,0)$ is reached. Our main result is that there is a phase transition: as $n\to\infty$, the expected reward under the optimal policy approaches a constant $c_{G}>0$ when $(n_{e}/n:e\in E)$ converges to a point in the interior of a certain convex set $\mathcal{R}_{G}$, and goes to $0$ exponentially when $(n_{e}/n:e\in E)$ is bounded away from $\mathcal{R}_{G}$. We also obtain estimates in the near-critical region, that is when $(n_{e}/n:e\in E)$ lies close to $\partial\mathcal{R}_{G}$. We supply quantitative error bounds in our arguments.

#### Article information

Source
Ann. Appl. Probab., Volume 27, Number 4 (2017), 2098-2129.

Dates
Revised: September 2016
First available in Project Euclid: 30 August 2017

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1504080027

Digital Object Identifier
doi:10.1214/16-AAP1250

Mathematical Reviews number (MathSciNet)
MR3693521

Zentralblatt MATH identifier
06803458

#### Citation

Járai, Antal A. Phase transition in a sequential assignment problem on graphs. Ann. Appl. Probab. 27 (2017), no. 4, 2098--2129. doi:10.1214/16-AAP1250. https://projecteuclid.org/euclid.aoap/1504080027

#### References

• [1] Albright, C. and Derman, C. (1972). Asymptotic optimal policies for the stochastic sequential assignment problem. Manage. Sci. 19 46–51.
• [2] Bollobás, B. (1998). Modern Graph Theory. Graduate Texts in Mathematics 184. Springer, New York.
• [3] Derman, C., Lieberman, G. J. and Ross, S. M. (1972). A sequential stochastic assignment problem. Manage. Sci. 18 349–355.
• [4] Grimmett, G. (2010). Probability on Graphs. Institute of Mathematical Statistics Textbooks 1. Cambridge Univ. Press, Cambridge.
• [5] Grimmett, G. R. and Stirzaker, D. R. (1992). Probability and Random Processes, 2nd ed. Oxford Univ. Press, New York.
• [6] Járai, A. A. (2016). The dice-and-numbers game. Gaz. Math. 100 410–419.
• [7] Jia, T., Liu, Y.-Y., Csóka, E., Pósfai, M., Slotine, J.-J. and Barabási, A.-L. (2013). Emergence of bimodality in controlling complex networks. Nat. Commun. 4 4.
• [8] Levin, D. A., Peres, Y. and Wilmer, E. L. (2009). Markov Chains and Mixing Times. Amer. Math. Soc., Providence, RI.
• [9] Nepusz, T. and Vicsek, T. (2012). Controlling edge dynamics in complex networks. Nature Physics 8 568–573.
• [10] Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V. and Mishchenko, E. F. (1962). The Mathematical Theory of Optimal Processes. Wiley, New York.
• [11] Puterman, M. L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York.
• [12] Ross, S. (1983). Introduction to Stochastic Dynamic Programming. Academic Press, New York.
• [13] Sun, J. and Motter, A. E. (2013). Controllability transition and nonlocality in network control. Phys. Rev. Lett. 110 208701.
• [14] Williams, D. (1991). Probability with Martingales. Cambridge Univ. Press, Cambridge.