The Annals of Applied Probability

Phase transition in a sequential assignment problem on graphs

Antal A. Járai

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


We study the following sequential assignment problem on a finite graph $G=(V,E)$. Each edge $e\in E$ starts with an integer value $n_{e}\ge0$, and we write $n=\sum_{e\in E}n_{e}$. At time $t$, $1\le t\le n$, a uniformly random vertex $v\in V$ is generated, and one of the edges $f$ incident with $v$ must be selected. The value of $f$ is then decreased by $1$. There is a unit final reward if the configuration $(0,\ldots,0)$ is reached. Our main result is that there is a phase transition: as $n\to\infty$, the expected reward under the optimal policy approaches a constant $c_{G}>0$ when $(n_{e}/n:e\in E)$ converges to a point in the interior of a certain convex set $\mathcal{R}_{G}$, and goes to $0$ exponentially when $(n_{e}/n:e\in E)$ is bounded away from $\mathcal{R}_{G}$. We also obtain estimates in the near-critical region, that is when $(n_{e}/n:e\in E)$ lies close to $\partial\mathcal{R}_{G}$. We supply quantitative error bounds in our arguments.

Article information

Ann. Appl. Probab., Volume 27, Number 4 (2017), 2098-2129.

Received: July 2015
Revised: September 2016
First available in Project Euclid: 30 August 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K99: None of the above, but in this section
Secondary: 91A60: Probabilistic games; gambling [See also 60G40] 90C40: Markov and semi-Markov decision processes

Phase transition critical phenomenon stochastic sequential assignment Markov decision process stochastic dynamic programming discrete stochastic optimal control


Járai, Antal A. Phase transition in a sequential assignment problem on graphs. Ann. Appl. Probab. 27 (2017), no. 4, 2098--2129. doi:10.1214/16-AAP1250.

Export citation


  • [1] Albright, C. and Derman, C. (1972). Asymptotic optimal policies for the stochastic sequential assignment problem. Manage. Sci. 19 46–51.
  • [2] Bollobás, B. (1998). Modern Graph Theory. Graduate Texts in Mathematics 184. Springer, New York.
  • [3] Derman, C., Lieberman, G. J. and Ross, S. M. (1972). A sequential stochastic assignment problem. Manage. Sci. 18 349–355.
  • [4] Grimmett, G. (2010). Probability on Graphs. Institute of Mathematical Statistics Textbooks 1. Cambridge Univ. Press, Cambridge.
  • [5] Grimmett, G. R. and Stirzaker, D. R. (1992). Probability and Random Processes, 2nd ed. Oxford Univ. Press, New York.
  • [6] Járai, A. A. (2016). The dice-and-numbers game. Gaz. Math. 100 410–419.
  • [7] Jia, T., Liu, Y.-Y., Csóka, E., Pósfai, M., Slotine, J.-J. and Barabási, A.-L. (2013). Emergence of bimodality in controlling complex networks. Nat. Commun. 4 4.
  • [8] Levin, D. A., Peres, Y. and Wilmer, E. L. (2009). Markov Chains and Mixing Times. Amer. Math. Soc., Providence, RI.
  • [9] Nepusz, T. and Vicsek, T. (2012). Controlling edge dynamics in complex networks. Nature Physics 8 568–573.
  • [10] Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V. and Mishchenko, E. F. (1962). The Mathematical Theory of Optimal Processes. Wiley, New York.
  • [11] Puterman, M. L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York.
  • [12] Ross, S. (1983). Introduction to Stochastic Dynamic Programming. Academic Press, New York.
  • [13] Sun, J. and Motter, A. E. (2013). Controllability transition and nonlocality in network control. Phys. Rev. Lett. 110 208701.
  • [14] Williams, D. (1991). Probability with Martingales. Cambridge Univ. Press, Cambridge.