The Annals of Applied Probability

Randomized and backward SDE representation for optimal control of non-Markovian SDEs

Marco Fuhrman and Huyên Pham

Full-text: Open access

Abstract

We study optimal stochastic control problems for non-Markovian stochastic differential equations (SDEs) where the drift, diffusion coefficients and gain functionals are path-dependent, and importantly we do not make any ellipticity assumptions on the SDE. We develop a control randomization approach and prove that the value function can be reformulated under a family of dominated measures on an enlarged filtered probability space. This value function is then characterized by a backward SDE with nonpositive jumps under a single probability measure, which can be viewed as a path-dependent version of the Hamilton–Jacobi–Bellman equation, and an extension to $G$-expectation.

Article information

Source
Ann. Appl. Probab., Volume 25, Number 4 (2015), 2134-2167.

Dates
Received: November 2013
Revised: April 2014
First available in Project Euclid: 21 May 2015

Permanent link to this document
https://projecteuclid.org/euclid.aoap/1432212439

Digital Object Identifier
doi:10.1214/14-AAP1045

Mathematical Reviews number (MathSciNet)
MR3349004

Zentralblatt MATH identifier
1322.60087

Subjects
Primary: 60H10: Stochastic ordinary differential equations [See also 34F05]
Secondary: 93E20: Optimal stochastic control

Keywords
Non-Markovian controlled SDEs randomization of controls dominated measures backward stochastic differential equations

Citation

Fuhrman, Marco; Pham, Huyên. Randomized and backward SDE representation for optimal control of non-Markovian SDEs. Ann. Appl. Probab. 25 (2015), no. 4, 2134--2167. doi:10.1214/14-AAP1045. https://projecteuclid.org/euclid.aoap/1432212439


Export citation

References

  • [1] Bertsekas, D. P. and Shreve, S. E. (1978). Stochastic Optimal Control: The Discrete Time Case. Mathematics in Science and Engineering 139. Academic Press, New York.
  • [2] Bouchard, B. (2009). A stochastic target formulation for optimal switching problems in finite horizon. Stochastics 81 171–197.
  • [3] Cheridito, P., Soner, H. M., Touzi, N. and Victoir, N. (2007). Second-order backward stochastic differential equations and fully nonlinear parabolic PDEs. Comm. Pure Appl. Math. 60 1081–1110.
  • [4] Crandall, M. G., Ishii, H. and Lions, P.-L. (1992). User’s guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc. (N.S.) 27 1–67.
  • [5] Denis, L., Hu, M. and Peng, S. (2011). Function spaces and capacity related to a sublinear expectation: Application to $G$-Brownian motion paths. Potential Anal. 34 139–161.
  • [6] Ekren, I., Keller, C., Touzi, N. and Zhang, J. (2014). On viscosity solutions of path dependent PDEs. Ann. Probab. 42 204–236.
  • [7] El Karoui, N. (1981). Les aspects probabilistes du contrôle stochastique. In Ninth Saint Flour Probability Summer School—1979 (Saint Flour, 1979). Lecture Notes in Math. 876 73–238. Springer, Berlin.
  • [8] El Karoui, N., Nguyen, D. H. and Jeanblanc-Picqué, M. (1988). Existence of an optimal Markovian filter for the control under partial observations. SIAM J. Control Optim. 26 1025–1061.
  • [9] El Karoui, N. and Tan, X. (2013). Capacities measurable selection and dynamic programming Part II: Application in stochastic control problems. Preprint. Available at arXiv:1310.3364.
  • [10] Fabbri, G., Gozzi, F. and Świȩch, A. (2015). Stochastic Optimal Control in Infinite Dimensions: Dynamic Programming and HJB Equations. Book in preparation.
  • [11] Fleming, W. H. (1977). Generalized solutions in optimal stochastic control. In Differential Games and Control Theory, II (Proc. 2nd Conf., Univ. Rhode Island, Kingston, R.I., 1976) Lecture Notes in Pure and Appl. Math. 30 147–165. Dekker, New York.
  • [12] Fleming, W. H. and Soner, H. M. (2006). Controlled Markov Processes and Viscosity Solutions, 2nd ed. Stochastic Modelling and Applied Probability 25. Springer, New York.
  • [13] He, S. W., Wang, J. G. and Yan, J. A. (1992). Semimartingale Theory and Stochastic Calculus. CRC Press, Boca Raton, FL.
  • [14] Jacod, J. (1975). Multivariate point processes: Predictable projection, Radon–Nikodým derivatives, representation of martingales. Z. Wahrsch. Verw. Gebiete 31 235–253.
  • [15] Kharroubi, I., Langrené, N. and Pham, H. (2014). Discrete time approximation of fully nonlinear HJB equations via BSDEs with nonpositive jumps. Ann. Appl. Probab. To appear. Available at arXiv:1311.4505.
  • [16] Kharroubi, I., Ma, J., Pham, H. and Zhang, J. (2010). Backward SDEs with constrained jumps and quasi-variational inequalities. Ann. Probab. 38 794–840.
  • [17] Kharroubi, I. and Pham, H. (2014). Feynman–Kac representation for Hamilton–Jacobi–Bellman IPDE. Ann. Probab. To appear.
  • [18] Krylov, N. V. (1980). Controlled Diffusion Processes. Applications of Mathematics 14. Springer, Berlin.
  • [19] Nutz, M. (2012). A quasi-sure approach to the control of non-Markovian stochastic differential equations. Electron. J. Probab. 17 23.
  • [20] Nutz, M. and van Handel, R. (2013). Constructing sublinear expectations on path space. Stochastic Process. Appl. 123 3100–3121.
  • [21] Pardoux, É. and Peng, S. (1992). Backward stochastic differential equations and quasilinear parabolic partial differential equations. In Stochastic Partial Differential Equations and Their Applications (Charlotte, NC, 1991) (B. Rozovskii and R. Sowers, eds.). Lecture Notes in Control and Inform. Sci. 176 200–217. Springer, Berlin.
  • [22] Peng, S. (2007). $G$-expectation, $G$-Brownian motion and related stochastic calculus of Itô type. In Stochastic Analysis and Applications. Abel Symp. 2 541–567. Springer, Berlin.
  • [23] Peng, S. (2011). Note on viscosity solutions of path-dependent PDE and $G$-martingales. Preprint. Available at arXiv:1106.1144v1.
  • [24] Pham, H. (2009). Continuous-Time Stochastic Control and Optimization with Financial Applications. Stochastic Modelling and Applied Probability 61. Springer, Berlin.
  • [25] Protter, P. and Shimbo, K. (2008). No arbitrage and general semimartingales. In Markov Processes and Related Topics: A Festschrift for Thomas G. Kurtz. Inst. Math. Stat. Collect. 4 267–283. IMS, Beachwood, OH.
  • [26] Rogers, L. C. G. and Williams, D. (2000). Diffusions, Markov Processes, and Martingales: Itô Calculus, Vol. 2. Cambridge Univ. Press, Cambridge.
  • [27] Soner, H. M., Touzi, N. and Zhang, J. (2012). Wellposedness of second order backward SDEs. Probab. Theory Related Fields 153 149–190.
  • [28] Tang, S. and Zhang, F. (2013). Path-dependent optimal stochastic control and viscosity solution of associated Bellman equations. Preprint. Available at arxiv:1210.2078.
  • [29] Tang, S. J. and Li, X. J. (1994). Necessary conditions for optimal control of stochastic systems with random jumps. SIAM J. Control Optim. 32 1447–1475.
  • [30] Yong, J. and Zhou, X. Y. (1999). Stochastic Controls: Hamiltonian Systems and HJB Equations. Applications of Mathematics (New York) 43. Springer, New York.
  • [31] Zabczyk, J. (1996). Chance and Decision: Stochastic Control in Discrete Time. Scuola Normale Superiore, Pisa.