The Annals of Applied Probability

Numerical method for optimal stopping of piecewise deterministic Markov processes

Benoîte de Saporta, François Dufour, and Karen Gonzalez

Full-text: Open access


We propose a numerical method to approximate the value function for the optimal stopping problem of a piecewise deterministic Markov process (PDMP). Our approach is based on quantization of the post jump location—inter-arrival time Markov chain naturally embedded in the PDMP, and path-adapted time discretization grids. It allows us to derive bounds for the convergence rate of the algorithm and to provide a computable ε-optimal stopping time. The paper is illustrated by a numerical example.

Article information

Ann. Appl. Probab., Volume 20, Number 5 (2010), 1607-1637.

First available in Project Euclid: 25 August 2010

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 93E20: Optimal stochastic control
Secondary: 93E03: Stochastic systems, general 60J25: Continuous-time Markov processes on general state spaces

Optimal stopping piecewise deterministic Markov processes quantization numerical method dynamic programming


de Saporta, Benoîte; Dufour, François; Gonzalez, Karen. Numerical method for optimal stopping of piecewise deterministic Markov processes. Ann. Appl. Probab. 20 (2010), no. 5, 1607--1637. doi:10.1214/09-AAP667.

Export citation


  • [1] Bally, V. and Pagès, G. (2003). A quantization algorithm for solving multi-dimensional discrete-time optimal stopping problems. Bernoulli 9 1003–1049.
  • [2] Bally, V., Pagès, G. and Printems, J. (2005). A quantization tree method for pricing and hedging multidimensional American options. Math. Finance 15 119–168.
  • [3] Costa, O. L. V. and Davis, M. H. A. (1988). Approximations for optimal stopping of a piecewise-deterministic process. Math. Control Signals Systems 1 123–146.
  • [4] Costa, O. L. V. and Dufour, F. (2008). Stability and ergodicity of piecewise deterministic Markov processes. SIAM J. Control Optim. 47 1053–1077.
  • [5] Costa, O. L. V., Raymundo, C. A. B. and Dufour, F. (2000). Optimal stopping with continuous control of piecewise deterministic Markov processes. Stochastics Stochastics Rep. 70 41–73.
  • [6] Davis, M. H. A. (1993). Markov Models and Optimization. Monographs on Statistics and Applied Probability 49. Chapman and Hall, London.
  • [7] Dufour, F. and Costa, O. L. V. (1999). Stability of piecewise-deterministic Markov processes. SIAM J. Control Optim. 37 1483–1502.
  • [8] Elliott, R. J. (1982). Stochastic Calculus and Applications. Applications of Mathematics (New York) 18. Springer, New York.
  • [9] Ga̧tarek, D. (1991). On first-order quasi-variational inequalities with integral terms. Appl. Math. Optim. 24 85–98.
  • [10] Gray, R. M. and Neuhoff, D. L. (1998). Quantization. IEEE Trans. Inform. Theory 44 2325–2383.
  • [11] Gugerli, U. S. (1986). Optimal stopping of a piecewise-deterministic Markov process. Stochastics 19 221–236.
  • [12] Kushner, H. J. (1977). Probability Methods for Approximations in Stochastic Control and for Elliptic Equations. Mathematics in Science and Engineering 129. Academic Press, New York.
  • [13] Lenhart, S. and Liao, Y. C. (1985). Integro-differential equations associated with optimal stopping time of a piecewise-deterministic process. Stochastics 15 183–207.
  • [14] Pagès, G. (1998). A space quantization method for numerical integration. J. Comput. Appl. Math. 89 1–38.
  • [15] Pagès, G. and Pham, H. (2005). Optimal quantization methods for nonlinear filtering with discrete-time observations. Bernoulli 11 893–932.
  • [16] Pagès, G., Pham, H. and Printems, J. (2004). An optimal Markovian quantization algorithm for multi-dimensional stochastic control problems. Stoch. Dyn. 4 501–545.
  • [17] Pagès, G., Pham, H. and Printems, J. (2004). Optimal quantization methods and applications to numerical problems in finance. In Handbook of Computational and Numerical Methods in Finance 253–297. Birkhäuser, Boston, MA.