Electronic Communications in Probability

Stochastic Perron's method for optimal control problems with state constraints

Dmitry Rokhlin

Full-text: Open access

Abstract

We apply the stochastic Perron method of Bayraktar and Sîrbu to a general infinite horizon optimal control problem, where the state $X$ is a controlled diffusion process, and the state constraint is described by a closed set. We prove that the value function $v$ is bounded from below (resp., from above) by a viscosity supersolution (resp., subsolution) of the related state constrained problem for the Hamilton-Jacobi-Bellman equation. In the case of a smooth domain, under some additional assumptions, these estimates allow to identify $v$ with a unique continuous constrained viscosity solution of this equation.

Article information

Source
Electron. Commun. Probab., Volume 19 (2014), paper no. 73, 15 pp.

Dates
Accepted: 20 October 2014
First available in Project Euclid: 7 June 2016

Permanent link to this document
https://projecteuclid.org/euclid.ecp/1465316775

Digital Object Identifier
doi:10.1214/ECP.v19-3616

Mathematical Reviews number (MathSciNet)
MR3274519

Zentralblatt MATH identifier
1310.93085

Subjects
Primary: 93E20: Optimal stochastic control
Secondary: 49L25: Viscosity solutions 60H30: Applications of stochastic analysis (to PDE, etc.)

Keywords
Stochastic Perron's method State constraints Viscosity solution Comparison result

Rights
This work is licensed under a Creative Commons Attribution 3.0 License.

Citation

Rokhlin, Dmitry. Stochastic Perron's method for optimal control problems with state constraints. Electron. Commun. Probab. 19 (2014), paper no. 73, 15 pp. doi:10.1214/ECP.v19-3616. https://projecteuclid.org/euclid.ecp/1465316775


Export citation

References

  • Bardi, Martino; Goatin, Paola. Invariant sets for controlled degenerate diffusions: a viscosity solutions approach. Stochastic analysis, control, optimization and applications, 191–208, Systems Control Found. Appl., Birkhauser Boston, Boston, MA, 1999.
  • Bardi, Martino; Jensen, Robert. A geometric characterization of viable sets for controlled degenerate diffusions. Calculus of variations, nonsmooth analysis and related topics. Set-Valued Anal. 10 (2002), no. 2-3, 129–141.
  • Barles, G.; Rouy, E. A strong comparison result for the Bellman equation arising in stochastic exit time control problems and its applications. Comm. Partial Differential Equations 23 (1998), no. 11-12, 1995–2033.
  • Bass, Richard F. Stochastic processes. Cambridge Series in Statistical and Probabilistic Mathematics, 33. Cambridge University Press, Cambridge, 2011. xvi+390 pp. ISBN: 978-1-107-00800-7
  • Bayraktar, Erhan; Sirbu, Mihai. Stochastic Perron's method and verification without smoothness using viscosity comparison: the linear case. Proc. Amer. Math. Soc. 140 (2012), no. 10, 3645–3654.
  • Bayraktar, Erhan; Sirbu, Mihai. Stochastic Perron's method for Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim. 51 (2013), no. 6, 4274–4294.
  • Bayraktar, Erhan; Sirbu, Mihai. Stochastic Perron's method and verification without smoothness using viscosity comparison: obstacle problems and Dynkin games. Proc. Amer. Math. Soc. 142 (2014), no. 4, 1399–1412.
  • E. Bayraktar and Y. Zhang, phStochastic Perron's method for the probability of lifetime ruin problem under transaction costs, Preprint arXiv:1404.7406v1 [math.OC], 24 pages, 2014.
  • Bichteler, Klaus. Stochastic integration and $L^{p}$-theory of semimartingales. Ann. Probab. 9 (1981), no. 1, 49–89.
  • Bouchard, Bruno; Nutz, Marcel. Weak dynamic programming for generalized state constraints. SIAM J. Control Optim. 50 (2012), no. 6, 3344–3373.
  • Buckdahn, R.; Goreac, D.; Quincampoix, M. Stochastic optimal control and linear programming approach. Appl. Math. Optim. 63 (2011), no. 2, 257–276.
  • Crandall, Michael G.; Ishii, Hitoshi; Lions, Pierre-Louis. User's guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc. (N.S.) 27 (1992), no. 1, 1–67.
  • Dellacherie, Claude; Meyer, Paul-Andre. Probabilities and potential. North-Holland Mathematics Studies, 29. North-Holland Publishing Co., Amsterdam-New York; North-Holland Publishing Co., Amsterdam-New York, 1978. viii+189 pp. ISBN: 0-7204-0701-X
  • Gilbarg, David; Trudinger, Neil S. Elliptic partial differential equations of second order. Reprint of the 1998 edition. Classics in Mathematics. Springer-Verlag, Berlin, 2001. xiv+517 pp. ISBN: 3-540-41160-7
  • He, Sheng Wu; Wang, Jia Gang; Yan, Jia An. Semimartingale theory and stochastic calculus. Kexue Chubanshe (Science Press), Beijing; CRC Press, Boca Raton, FL, 1992. xiv+546 pp. ISBN: 7-03-003066-4
  • Ikeda, Nobuyuki; Watanabe, Shinzo. Stochastic differential equations and diffusion processes. Second edition. North-Holland Mathematical Library, 24. North-Holland Publishing Co., Amsterdam; Kodansha, Ltd., Tokyo, 1989. xvi+555 pp. ISBN: 0-444-87378-3
  • Ishii, Hitoshi. Perron's method for Hamilton-Jacobi equations. Duke Math. J. 55 (1987), no. 2, 369–384.
  • Ishii, Hitoshi; Loreti, Paola. A class of stochastic optimal control problems with state constraint. Indiana Univ. Math. J. 51 (2002), no. 5, 1167–1196.
  • Karandikar, Rajeeva L. Pathwise solutions of stochastic differential equations. Sankhyā Ser. A 43 (1981), no. 2, 121–132.
  • Karandikar, Rajeeva L. On pathwise stochastic integration. Stochastic Process. Appl. 57 (1995), no. 1, 11–18.
  • Katsoulakis, Markos A. Viscosity solutions of second order fully nonlinear elliptic equations with state constraints. Indiana Univ. Math. J. 43 (1994), no. 2, 493–519.
  • Krylov, N. V. Controlled diffusion processes. Translated from the Russian by A. B. Aries. Applications of Mathematics, 14. Springer-Verlag, New York-Berlin, 1980. xii+308 pp. ISBN: 0-387-90461-1
  • Lasry, J.-M.; Lions, P.-L. Nonlinear elliptic equations with singular boundary conditions and stochastic control with state constraints. I. The model problem. Math. Ann. 283 (1989), no. 4, 583–630.
  • Rokhlin, Dmitry B. Verification by stochastic Perron's method in stochastic exit time control problems. J. Math. Anal. Appl. 419 (2014), no. 1, 433–446.
  • Sirbu, Mihai. Stochastic Perron's method and elementary strategies for zero-sum differential games. SIAM J. Control Optim. 52 (2014), no. 3, 1693–1711.
  • bysame, phAsymptotic Perron's method and simple Markov strategies in stochastic games and control, Preprint arXiv:1402.7030 [math.OC], 12 pages, 2014.
  • Soner, Halil Mete. Optimal control with state-space constraint. I. SIAM J. Control Optim. 24 (1986), no. 3, 552–561.
  • Stroock, Daniel W. Probability theory. An analytic view. Second edition. Cambridge University Press, Cambridge, 2011. xxii+527 pp. ISBN: 978-0-521-13250-3
  • Stroock, Daniel W.; Varadhan, S. R. Srinivasa. Multidimensional diffusion processes. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], 233. Springer-Verlag, Berlin-New York, 1979. xii+338 pp. ISBN: 3-540-90353-4