Electronic Journal of Probability

On time reversal of piecewise deterministic Markov processes

Andreas Löpker and Zbigniew Palmowski

Full-text: Open access


We study the time reversal of a general Piecewise Deterministic Markov Process (PDMP). The time reversed process is defined as $X_{(T-t)-}$, where $T$ is some given time and $X_t$ is a stationary PDMP. We obtain the parameters of the reversed process, like the jump intensity and the jump measure.

Article information

Electron. J. Probab., Volume 18 (2013), paper no. 13, 29 pp.

Accepted: 23 January 2013
First available in Project Euclid: 4 June 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60J25: Continuous-time Markov processes on general state spaces
Secondary: 60J27: Continuous-time Markov processes on discrete state spaces

Piecewise Deterministic Markov Processes time reversal Stationary distribution

This work is licensed under a Creative Commons Attribution 3.0 License.


Löpker, Andreas; Palmowski, Zbigniew. On time reversal of piecewise deterministic Markov processes. Electron. J. Probab. 18 (2013), paper no. 13, 29 pp. doi:10.1214/EJP.v18-1958. https://projecteuclid.org/euclid.ejp/1465064238

Export citation


  • Albrecher, Hansjörg; Thonhauser, Stefan. Optimality results for dividend problems in insurance. Rev. R. Acad. Cienc. Exactas Fís. Nat. Ser. A Math. RACSAM 103 (2009), no. 2, 295–320.
  • E. Altman, K. Avrachenkov, A. Kherani, and B. Prabhu. Performance Analysis and Stochastic Stability of Congestion Control Protocols, Tech. Report RR-5262, INRIA, Sophia-Antipolis, France, July 2004.
  • Anick, D.; Mitra, D.; Sondhi, M. M. Stochastic theory of a data-handling system with multiple sources. Bell System Tech. J. 61 (1982), no. 8, 1871–1894.
  • Asmussen, Søren. Stationary distributions for fluid flow models with or without Brownian noise. Comm. Statist. Stochastic Models 11 (1995), no. 1, 21–49.
  • Asmussen, Søren. Applied probability and queues. Second edition. Applications of Mathematics (New York), 51. Stochastic Modelling and Applied Probability. Springer-Verlag, New York, 2003. xii+438 pp. ISBN: 0-387-00211-1
  • Baccelli, Francois; Kim, Ki Baek; McDonald, David R. Equilibria of a class of transport equations arising in congestion control. Queueing Syst. 55 (2007), no. 1, 1–8.
  • Bar-Lev, Shaul K.; Parlar, Mahmut; Perry, David. On the EOQ model with inventory-level-dependent demand rate and random yield. Oper. Res. Lett. 16 (1994), no. 3, 167–176.
  • J.-B. Bardet, A. Christen, A. Guillin, F. Malrieu, and P.-A. Zitt, phTotal variation estimates for the TCP process, ArXiv e-prints (2011).
  • Bekker, R.; Borst, S. C.; Boxma, O. J.; Kella, O. Queues with workload-dependent arrival and service rates. Queueing Syst. 46 (2004), no. 3-4, 537–556.
  • Billingsley, Patrick. Convergence of probability measures. John Wiley & Sons, Inc., New York-London-Sydney 1968 xii+253 pp.
  • Borovkov, K.; Novikov, A. On a piece-wise deterministic Markov process model. Statist. Probab. Lett. 53 (2001), no. 4, 421–428.
  • Borovkov, K.; Vere-Jones, D. Explicit formulae for stationary distributions of stress release processes. J. Appl. Probab. 37 (2000), no. 2, 315–321.
  • Boxma, Onno; Kaspi, Haya; Kella, Offer; Perry, David. On/off storage systems with state-dependent input, output, and switching rates. Probab. Engrg. Inform. Sci. 19 (2005), no. 1, 1–14.
  • Boxma, Onno; Perry, David; Stadje, Wolfgang; Zacks, Shelemyahu. A Markovian growth-collapse model. Adv. in Appl. Probab. 38 (2006), no. 1, 221–243.
  • Brockwell, P. J. Stationary distributions for dams with additive input and content-dependent release rate. Advances in Appl. Probability 9 (1977), no. 3, 645–663.
  • Browne, Sid; Sigman, Karl. Work-modulated queues with applications to storage processes. J. Appl. Probab. 29 (1992), no. 3, 699–712.
  • Çinlar, E.; Pinsky, M. A stochastic integral in storage theory. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 17 (1971), 227–240.
  • Chafaï, Djalil; Malrieu, Florent; Paroux, Katy. On the long time behavior of the TCP window size process. Stochastic Process. Appl. 120 (2010), no. 8, 1518–1534.
  • Chung, K. L.; Walsh, John B. To reverse a Markov process. Acta Math. 123 1969 225–251.
  • Costa, O. L. V. Stationary distributions for piecewise-deterministic Markov processes. J. Appl. Probab. 27 (1990), no. 1, 60–73.
  • Dassios, A.; Embrechts, P. Martingales and insurance risk. Comm. Statist. Stochastic Models 5 (1989), no. 2, 181–217.
  • Dassios, Angelos; Jang, Jiwook. The distribution of the interval between events of a Cox process with shot noise intensity. J. Appl. Math. Stoch. Anal. 2008, Art. ID 367170, 14 pp.
  • Davis, M. H. A. Piecewise-deterministic Markov processes: a general class of nondiffusion stochastic models. With discussion. J. Roy. Statist. Soc. Ser. B 46 (1984), no. 3, 353–388.
  • Davis, M. H. A. Markov models and optimization. Monographs on Statistics and Applied Probability, 49. Chapman & Hall, London, 1993. xiv+295 pp. ISBN: 0-412-31410-X
  • Dufour, François; Costa, Oswaldo L. V. Stability of piecewise-deterministic Markov processes. SIAM J. Control Optim. 37 (1999), no. 5, 1483–1502 (electronic).
  • Dumas, Vincent; Guillemin, Fabrice; Robert, Philippe. A Markovian analysis of additive-increase multiplicative-decrease algorithms. Adv. in Appl. Probab. 34 (2002), no. 1, 85–111.
  • Embrechts, Paul; Schmidli, Hanspeter. Ruin estimation for a general insurance risk model. Adv. in Appl. Probab. 26 (1994), no. 2, 404–422.
  • Faggionato, A.; Gabrielli, D.; Ribezzi Crivellari, M. Non-equilibrium thermodynamics of piecewise deterministic Markov processes. J. Stat. Phys. 137 (2009), no. 2, 259–304.
  • Hansen, Lars Peter; Scheinkman, José Alexandre. Back to the future: generating moment implications for continuous-time Markov processes. Econometrica 63 (1995), no. 4, 767–804.
  • Harrison, J. Michael; Resnick, Sidney I. The stationary distribution and first exit probabilities of a storage process with general release rule. Math. Oper. Res. 1 (1976), no. 4, 347–358.
  • Harrison, J. Michael; Resnick, Sidney I. The recurrence classification of risk and storage processes. Math. Oper. Res. 3 (1978), no. 1, 57–66.
  • P.G. Harrison and N. Thomas. Product-form solution in PEPA via the reversed process, Network Performance Engineering (D. Kouvatsos, ed.), Lecture Notes in Computer Science, vol. 5233, Springer Berlin Heidelberg, 2011, pp. 343–356.
  • Hartman, Philip. Ordinary differential equations. Reprint of the second edition. Birkhäuser, Boston, Mass., 1982. xv+612 pp. ISBN: 3-7643-3068-6
  • Jacobsen, Martin. Point process theory and applications. Marked point and piecewise deterministic processes. Probability and its Applications. Birkhäuser Boston, Inc., Boston, MA, 2006. xii+328 pp. ISBN: 978-0-8176-4215-0; 0-8176-4215-3
  • Jacod, Jean; Protter, Philip. Time reversal on Lévy processes. Ann. Probab. 16 (1988), no. 2, 620–641.
  • Jacod, J.; Skorohod, A. V. Jumping filtrations and martingales with finite variation. Séminaire de Probabilités, XXVIII, 21–35, Lecture Notes in Math., 1583, Springer, Berlin, 1994.
  • Kelly, Frank P. Reversibility and stochastic networks. Wiley Series in Probability and Mathematical Statistics. John Wiley & Sons, Ltd., Chichester, 1979. viii+230 pp. ISBN: 0-471-27601-4
  • Kolmogoroff, A. Zur Theorie der Markoffschen Ketten. (German) Math. Ann. 112 (1936), no. 1, 155–160.
  • bysame, Zur Umkehrbarkeit der statistischen Naturgesetze., Math. Ann. 113 (1936), 766–772 (German).
  • Last, Günter. Ergodicity properties of stress release, repairable system and workload models. Adv. in Appl. Probab. 36 (2004), no. 2, 471–498.
  • Borovkov, K.; Last, G. On level crossings for a general class of piecewise-deterministic Markov processes. Adv. in Appl. Probab. 40 (2008), no. 3, 815–834.
  • Last, Günter; Szekli, Ryszard. Stochastic comparison of repairable systems by coupling. J. Appl. Probab. 35 (1998), no. 2, 348–370.
  • Löpker, Andreas; Stadje, Wolfgang. Hitting times and the running maximum of Markovian growth-collapse processes. J. Appl. Probab. 48 (2011), no. 2, 295–312.
  • Maulik, Krishanu; Zwart, Bert. Tail asymptotics for exponential functionals of Lévy processes. Stochastic Process. Appl. 116 (2006), no. 2, 156–177.
  • M. Miyazawa. Reversibility in Queueing Models, ArXiv e-prints (2012).
  • Nagasawa, Masao. Time reversions of Markov processes. Nagoya Math. J. 24 1964 177–204.
  • Nagasawa, Masao. Time reversal of Markov processes and relativistic quantum theory. Chaos Solitons Fractals 8 (1997), no. 11, 1711–1772.
  • Norris, J. R. Markov chains. Reprint of 1997 original. Cambridge Series in Statistical and Probabilistic Mathematics, 2. Cambridge University Press, Cambridge, 1998. xvi+237 pp. ISBN: 0-521-48181-3
  • Ogata, Y.; Vere-Jones, D. Inference for earthquake models: a self-correcting model. Stochastic Process. Appl. 17 (1984), no. 2, 337–347.
  • Palmowski, Zbigniew; Rolski, Tomasz. A technique for exponential change of measure for Markov processes. Bernoulli 8 (2002), no. 6, 767–785.
  • Rolski, Tomasz; Schmidli, Hanspeter; Schmidt, Volker; Teugels, Jozef. Stochastic processes for insurance and finance. Wiley Series in Probability and Statistics. John Wiley & Sons, Ltd., Chichester, 1999. xviii+654 pp. ISBN: 0-471-95925-1
  • M. Schäl, phSprungprozesse, Lecture Notes, University of Bonn (1997) (English).
  • Schäl, Manfred. On piecewise deterministic Markov control processes: control of jumps and of risk processes in insurance. The interplay between insurance, finance and control (Aarhus, 1997). Insurance Math. Econom. 22 (1998), no. 1, 75–91.
  • E. Schrödinger, phÃœber die Umkehrung der Naturgesetze., (1931) (German).
  • Tanaka, Hiroshi. Time reversal of random walks in one-dimension. Tokyo J. Math. 12 (1989), no. 1, 159–174.
  • E.A. Van Doorn and W.R.W. Scheinhardt, phAnalysis of birth-death fluid queues, Memorandum, University of Twente (1996).
  • Löpker, Andreas H.; van Leeuwaarden, Johan S. H. Transient moments of the TCP window size process. J. Appl. Probab. 45 (2008), no. 1, 163–175.
  • van Leeuwaarden, J. S. H.; Löpker, A. H.; Ott, T. J. TCP and iso-stationary transformations. Queueing Syst. 63 (2009), no. 1-4, 459–475.
  • D. Vere-Jones. On the variance properties of stress release models., Austral. J. Statist. 30A (1988), 123–135.
  • Walsh, John B. Time reversal and the completion of Markov processes. Invent. Math. 10 1970 57–81.
  • Weiss, Gideon. Time-reversibility of linear stochastic processes. J. Appl. Probability 12 (1975), no. 4, 831–836.
  • Wobst, Reinhard. On jump processes with drift. Dissertationes Math. (Rozprawy Mat.) 202 (1983), 51 pp.
  • Yeh, J. Lectures on real analysis. World Scientific Publishing Co., Inc., River Edge, NJ, 2000. xvi+548 pp. ISBN: 981-02-3936-X.
  • Zheng, Xiao Gu. Ergodic theorems for stress release processes. Stochastic Process. Appl. 37 (1991), no. 2, 239–258.