The Annals of Applied Probability

Optimal stopping problems for some Markov processes

Mamadou Cissé, Pierre Patie, and Etienne Tanré

Full-text: Open access


In this paper, we solve explicitly the optimal stopping problem with random discounting and an additive functional as cost of observations for a regular linear diffusion. We also extend the results to the class of one-sided regular Feller processes. This generalizes the result of Beibel and Lerche [Statist. Sinica 7 (1997) 93–108] and [Teor. Veroyatn. Primen. 45 (2000) 657–669] and Irles and Paulsen [Sequential Anal. 23 (2004) 297–316]. Our approach relies on a combination of techniques borrowed from potential theory and stochastic calculus. We illustrate our results by detailing some new examples ranging from linear diffusions to Markov processes of the spectrally negative type.

Article information

Ann. Appl. Probab. Volume 22, Number 3 (2012), 1243-1265.

First available in Project Euclid: 18 May 2012

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60G40: Stopping times; optimal stopping problems; gambling theory [See also 62L15, 91A60]
Secondary: 60J60: Diffusion processes [See also 58J65] 60J75: Jump processes

Optimal stopping problems Doob’s h-transform excessive functions Feller processes


Cissé, Mamadou; Patie, Pierre; Tanré, Etienne. Optimal stopping problems for some Markov processes. Ann. Appl. Probab. 22 (2012), no. 3, 1243--1265. doi:10.1214/11-AAP795.

Export citation


  • [1] Alili, L. and Kyprianou, A. E. (2005). Some remarks on first passage of Lévy processes, the American put and pasting principles. Ann. Appl. Probab. 15 2062–2080.
  • [2] Bally, V. and Stoica, L. (1987). A class of Markov processes which admit local times. Ann. Probab. 15 241–262.
  • [3] Baurdoux, E. J. (2007). Examples of optimal stopping via measure transformation for processes with one-sided jumps. Stochastics 79 303–307.
  • [4] Beibel, M. and Lerche, H. R. (1997). A new look at optimal stopping problems related to mathematical finance. Statist. Sinica 7 93–108.
  • [5] Beibel, M. and Lerche, H. R. (2000). A note on optimal stopping of regular diffusions under random discounting. Teor. Veroyatn. Primen. 45 657–669.
  • [6] Bertoin, J. (1996). Lévy Processes. Cambridge Tracts in Mathematics 121. Cambridge Univ. Press, Cambridge.
  • [7] Blumenthal, R. M. and Getoor, R. K. (1968). Markov Processes and Potential Theory. Pure and Applied Mathematics 29. Academic Press, New York.
  • [8] Borodin, A. N. and Salminen, P. (2002). Handbook of Brownian Motion—Facts and Formulae, 2nd ed. Birkhäuser, Basel.
  • [9] Carmona, P., Petit, F. and Yor, M. (1997). On the distribution and asymptotic results for exponential functionals of Lévy processes. In Exponential Functionals and Principal Values Related to Brownian Motion 73–130. Rev. Mat. Iberoam., Madrid.
  • [10] Dayanik, S. and Karatzas, I. (2003). On the optimal stopping problem for one-dimensional diffusions. Stochastic Process. Appl. 107 173–212.
  • [11] Dynkin, E. B. (1963). Optimal choice of the stopping moment of a Markov process. Dokl. Akad. Nauk SSSR 150 238–240.
  • [12] Graversen, S. E. and Peškir, G. (1997). On Wald-type optimal stopping for Brownian motion. J. Appl. Probab. 34 66–73.
  • [13] Hawkes, J. (1979). Potential theory of Lévy processes. Proc. Lond. Math. Soc. (3) 38 335–352.
  • [14] Irle, A. and Paulsen, V. (2004). Solving problems of optimal stopping with linear costs of observations. Sequential Anal. 23 297–316.
  • [15] Itô, K. and McKean, H. P. (1996). Diffusion Processes and Their Sample Paths. Springer, Berlin.
  • [16] Khoshnevisan, D., Salminen, P. and Yor, M. (2006). A note on a.s. finiteness of perpetual integral functionals of diffusions. Electron. Commun. Probab. 11 108–117 (electronic).
  • [17] Kramkov, D. O. and Mordetski, È. (1994). An integral option. Teor. Veroyatn. Primen. 39 201–211.
  • [18] Kyprianou, A. E. (2006). Introductory Lectures on Fluctuations of Lévy Processes with Applications. Springer, Berlin.
  • [19] Kyprianou, A. E. and Pistorius, M. R. (2003). Perpetual options and Canadization through fluctuation theory. Ann. Appl. Probab. 13 1077–1098.
  • [20] Lamperti, J. (1967). On random time substitutions and the Feller property. In Markov Processes and Potential Theory (Proc. Sympos. Math. Res. Center, Madison, Wis., 1967) 87–101. Wiley, New York.
  • [21] Lamperti, J. (1972). Semi-stable Markov processes. I. Z. Wahrsch. Verw. Gebiete 22 205–225.
  • [22] Lebedev, N. N. (1972). Special Functions and Their Applications. Dover, New York.
  • [23] Patie, P. (2008). q-invariant functions for some generalizations of the Ornstein–Uhlenbeck semigroup. ALEA Lat. Am. J. Probab. Math. Stat. 4 31–43.
  • [24] Patie, P. (2009). Exponential functional of a new family of Lévy processes and self-similar continuous state branching processes with immigration. Bull. Sci. Math. 133 355–382.
  • [25] Patie, P. (2009). Infinite divisibility of solutions to some self-similar integro-differential equations and exponential functionals of Lévy processes. Ann. Inst. Henri Poincaré Probab. Stat. 45 667–684.
  • [26] Patie, P. and Vigon, V. (2011). One-dimensional completely asymmetric Markov processes. Unpublished manuscript, ULB.
  • [27] Peskir, G. and Shiryaev, A. (2006). Optimal Stopping and Free-Boundary Problems. Birkhäuser, Basel.
  • [28] Pitman, J. and Yor, M. (1981). Bessel processes and infinitely divisible laws. In Stochastic Integrals (Proc. Sympos., Univ. Durham, Durham, 1980). Lecture Notes in Math. 851 285–370. Springer, Berlin.
  • [29] Prabhakar, T. R. (1971). A singular integral equation with a generalized Mittag Leffler function in the kernel. Yokohama Math. J. 19 7–15.
  • [30] Rivero, V. (2005). Recurrent extensions of self-similar Markov processes and Cramér’s condition. Bernoulli 11 471–509.
  • [31] Rogers, L. C. G. and Williams, D. (2000). Diffusions, Markov Processes, and Martingales. Vol. 1. Cambridge Univ. Press, Cambridge. Reprint of the second (1994) edition.
  • [32] Salminen, P. (1985). Optimal stopping of one-dimensional diffusions. Math. Nachr. 124 85–101.
  • [33] Shepp, L. A. (1967). A first passage problem for the Wiener process. Ann. Math. Statist. 38 1912–1914.
  • [34] Shiryayev, A. N. (1978). Optimal Stopping Rules. Springer, New York.
  • [35] Williams, D. (1974). Path decomposition and continuity of local time for one-dimensional diffusions. I. Proc. Lond. Math. Soc. (3) 28 738–768.