Journal of Applied Probability

Markov processes with restart

Konstantin Avrachenkov, Alexey Piunovskiy, and Yi Zhang

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

We consider a general homogeneous continuous-time Markov process with restarts. The process is forced to restart from a given distribution at time moments generated by an independent Poisson process. The motivation to study such processes comes from modeling human and animal mobility patterns, restart processes in communication protocols, and from application of restarting random walks in information retrieval. We provide a connection between the transition probability functions of the original Markov process and the modified process with restarts. We give closed-form expressions for the invariant probability measure of the modified process. When the process evolves on the Euclidean space, there is also a closed-form expression for the moments of the modified process. We show that the modified process is always positive Harris recurrent and exponentially ergodic with the index equal to (or greater than) the rate of restarts. Finally, we illustrate the general results by the standard and geometric Brownian motions.

Article information

Source
J. Appl. Probab., Volume 50, Number 4 (2013), 960-968.

Dates
First available in Project Euclid: 10 January 2014

Permanent link to this document
https://projecteuclid.org/euclid.jap/1389370093

Digital Object Identifier
doi:10.1239/jap/1389370093

Mathematical Reviews number (MathSciNet)
MR3161367

Zentralblatt MATH identifier
1295.60086

Subjects
Primary: 60J25: Continuous-time Markov processes on general state spaces
Secondary: 47D07: Markov semigroups and applications to diffusion processes {For Markov processes, see 60Jxx}

Keywords
Markov process with restart positive Harris recurrence exponential ergodicity standard and geometric Brownian motions

Citation

Avrachenkov, Konstantin; Piunovskiy, Alexey; Zhang, Yi. Markov processes with restart. J. Appl. Probab. 50 (2013), no. 4, 960--968. doi:10.1239/jap/1389370093. https://projecteuclid.org/euclid.jap/1389370093


Export citation

References

  • Alt, H., \et (1996). A method for obtaining randomized algorithms with small tail probabilities. Algorithmica 16, 543–547.
  • Avrachenkov, K. E., Filar, J. and Haviv, M. (2002). Singular perturbations of Markov chains and decision processes. In Handbook of Markov Decision Processes, eds E. A. Feinberg and A. Shwartz, Kluwer, Boston, MA, pp. 113–150.
  • Brin, S. and Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. Comput. Networks ISDN Systems 30, 107–117.
  • Chung, F. (2007). The heat kernel as the pagerank of a graph. Proc. Nat. Acad. Sci. 104, 19735–19740.
  • Down, D., Meyn, S. P. and Tweedie, R. L. (1995). Exponential and uniform ergodicity of Markov processes. Ann. Prob. 23, 1671–1691.
  • Feller, W. (1971). An Introduction to Probability Theory and Its Applications, Vol. II, 2nd edn. John Wiley, New York.
  • Glynn, P. W. (1994). Some topics in regenerative steady-state simulation. Acta Appl. Math. 34, 225–236.
  • González, M. C., Hidalgo, C. A. and Barabási, A.-L. (2008). Understanding individual human mobility patterns. Nature 453, 779–782.
  • Hernández-Lerma, O. and Lasserre, J.-B. (1996). Discrete-time Markov Control Processes, Springer, New York.
  • Krishnamurthy, B. and Rexford, J. (2001). Web Protocols and Practice: HTTP/1.1, Networking Protocols, Caching, and Traffic Measurement. Addison Wesley.
  • Kuznetsov, S. E. (1980). Any Markov process in a Borel space has a transition function. Theory Prob. Appl. 25, 384–388.
  • Luby, M., Sinclair, A. and Zuckerman, D. (1993). Optimal speedup of Las Vegas algorithms. Inf. Process. Lett. 47, 173–180.
  • Maurer, S. M. and Huberman, B. A. (2001). Restart strategies and Internet congestion. J. Econom. Dynamics Control 25, 641–654.
  • Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markov processes III. Forster-Lyapunov criteria for continuous-time processes. Adv. Appl. Prob. 25, 518–548.
  • Ross, S. M. (1996). Stochastic Processes, 2nd edn. John Wiley, New York.
  • Stevens, W. R. (1994). TCP/IP Illustrated, Volume 1: The Protocols. Addison Wesley.
  • Walsh, P. D., Boyer, D. and Crofoot, M. C. (2010). Monkey and cell-phone-user mobilities scale similarly. Nature Phys. 6, 929–930.