The Annals of Applied Probability

Discrete-time probabilistic approximation of path-dependent stochastic control problems

Xiaolu Tan

Full-text: Open access


We give a probabilistic interpretation of the Monte Carlo scheme proposed by Fahim, Touzi and Warin [Ann. Appl. Probab. 21 (2011) 1322–1364] for fully nonlinear parabolic PDEs, and hence generalize it to the path-dependent (or non-Markovian) case for a general stochastic control problem. A general convergence result is obtained by a weak convergence method in the spirit of Kushner and Dupuis [Numerical Methods for Stochastic Control Problems in Continuous Time (1992) Springer]. We also get a rate of convergence using the invariance principle technique as in Dolinsky [Electron. J. Probab. 17 (2012) 1–5], which is better than that obtained by viscosity solution method. Finally, by approximating the conditional expectations arising in the numerical scheme with simulation-regression method, we obtain an implementable scheme.

Article information

Ann. Appl. Probab., Volume 24, Number 5 (2014), 1803-1834.

First available in Project Euclid: 26 June 2014

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 65K99: None of the above, but in this section
Secondary: 93E20: Optimal stochastic control 93E25: Other computational methods

Numerical scheme path-dependent stochastic control weak convergence invariance principle


Tan, Xiaolu. Discrete-time probabilistic approximation of path-dependent stochastic control problems. Ann. Appl. Probab. 24 (2014), no. 5, 1803--1834. doi:10.1214/13-AAP963.

Export citation


  • [1] Barles, G. and Souganidis, P. E. (1991). Convergence of approximation schemes for fully nonlinear second order equations. Asymptot. Anal. 4 271–283.
  • [2] Bertsekas, D. P. and Shreve, S. E. (1978). Stochastic Optimal Control: The Discrete Time Case. Mathematics in Science and Engineering 139. Academic Press, New York.
  • [3] Bonnans, J. F., Ottenwaelter, É. and Zidani, H. (2004). A fast algorithm for the two dimensional HJB equation of stochastic control. M2AN Math. Model. Numer. Anal. 38 723–735.
  • [4] Bouchard, B. and Touzi, N. (2004). Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations. Stochastic Process. Appl. 111 175–206.
  • [5] Cheridito, P., Soner, H. M., Touzi, N. and Victoir, N. (2007). Second-order backward stochastic differential equations and fully nonlinear parabolic PDEs. Comm. Pure Appl. Math. 60 1081–1110.
  • [6] Debrabant, K. and Jakobsen, E. R. (2013). Semi-Lagrangian schemes for linear and fully non-linear diffusion equations. Math. Comp. 82 1433–1462.
  • [7] Dolinsky, Y. (2012). Numerical schemes for $G$-expectations. Electron. J. Probab. 17 1–15.
  • [8] Dolinsky, Y., Nutz, M. and Soner, H. M. (2012). Weak approximation of $G$-expectations. Stochastic Process. Appl. 122 664–675.
  • [9] El Karoui, N., Hu̇u̇ Nguyen, D. and Jeanblanc-Picqué, M. (1987). Compactification methods in the control of degenerate diffusions: Existence of an optimal control. Stochastics 20 169–219.
  • [10] El Karoui, N. and Tan, X. (2013). Capacities, measurable selection and dynamic programming. Preprint. Available at
  • [11] Fahim, A., Touzi, N. and Warin, X. (2011). A probabilistic numerical method for fully nonlinear parabolic PDEs. Ann. Appl. Probab. 21 1322–1364.
  • [12] Gobet, E. and Turkedjiev, P. (2013). Linear regression MDP scheme for discrete BSDEs under general conditions. Preprint. Available at
  • [13] Guyon, J. and Henry-Labordère, P. (2011). Uncertain volatility model: A Monte-Carlo approach. J. Comput. Finance 14 37–71.
  • [14] Jakobsen, E. R. (2004). On error bounds for approximation schemes for non-convex degenerate elliptic equations. BIT 44 269–285.
  • [15] Kushner, H. J. (1990). Numerical methods for stochastic control problems in continuous time. SIAM J. Control Optim. 28 999–1048.
  • [16] Kushner, H. J. and Dupuis, P. G. (1992). Numerical Methods for Stochastic Control Problems in Continuous Time. Applications of Mathematics (New York) 24. Springer, New York.
  • [17] Lemor, J.-P., Gobet, E. and Warin, X. (2006). Rate of convergence of an empirical regression method for solving generalized backward stochastic differential equations. Bernoulli 12 889–916.
  • [18] Peng, S. (2007). $G$-expectation, $G$-Brownian motion and related stochastic calculus of Itô type. In Stochastic Analysis and Applications. Abel Symp. 2 541–567. Springer, Berlin.
  • [19] Sakhanenko, A. I. (2000). A new way to obtain estimates in the invariance principle. In High Dimensional Probability II (E. Gine, D. M. Mason and J. A. Wellner, eds.) Progr. Probab. 47 221–243. Birkhäuser, Boston.
  • [20] Soner, H. M., Touzi, N. and Zhang, J. (2011). Quasi-sure stochastic analysis through aggregation. Electron. J. Probab. 16 1844–1879.
  • [21] Soner, H. M., Touzi, N. and Zhang, J. (2012). Wellposedness of second order backward SDEs. Probab. Theory Related Fields 153 149–190.
  • [22] Stroock, D. W. and Varadhan, S. R. S. (1979). Multidimensional Diffusion Processes. Grundlehren der Mathematischen Wissenschaften 233. Springer, Berlin.
  • [23] Tan, X. (2013). A splitting method for fully nonlinear degenerate parabolic PDEs. Electron. J. Probab. 18 1–24.
  • [24] Valadier, M. (1994). A course on Young measures. Rend. Istit. Mat. Univ. Trieste 26 349–394.
  • [25] Young, L. C. (1969). Lectures on the Calculus of Variations and Optimal Control Theory. W. B. Saunders, Philadelphia.
  • [26] Zhang, J. (2004). A numerical scheme for BSDEs. Ann. Appl. Probab. 14 459–488.