The Annals of Probability

Forward–backward stochastic differential equations and controlled McKean–Vlasov dynamics

René Carmona and François Delarue

Full-text: Open access

Abstract

The purpose of this paper is to provide a detailed probabilistic analysis of the optimal control of nonlinear stochastic dynamical systems of McKean–Vlasov type. Motivated by the recent interest in mean-field games, we highlight the connection and the differences between the two sets of problems. We prove a new version of the stochastic maximum principle and give sufficient conditions for existence of an optimal control. We also provide examples for which our sufficient conditions for existence of an optimal solution are satisfied. Finally we show that our solution to the control problem provides approximate equilibria for large stochastic controlled systems with mean-field interactions when subject to a common policy.

Article information

Source
Ann. Probab., Volume 43, Number 5 (2015), 2647-2700.

Dates
Received: March 2013
Revised: March 2014
First available in Project Euclid: 9 September 2015

Permanent link to this document
https://projecteuclid.org/euclid.aop/1441792295

Digital Object Identifier
doi:10.1214/14-AOP946

Mathematical Reviews number (MathSciNet)
MR3395471

Zentralblatt MATH identifier
1322.93103

Subjects
Primary: 93E20: Optimal stochastic control
Secondary: 60H10: Stochastic ordinary differential equations [See also 34F05] 60K35: Interacting random processes; statistical mechanics type models; percolation theory [See also 82B43, 82C43]

Keywords
Stochastic control McKean–Vlasov diffusion stochastic Pontryagin principle mean-field interaction mean-field forward–backward stochastic differential equation

Citation

Carmona, René; Delarue, François. Forward–backward stochastic differential equations and controlled McKean–Vlasov dynamics. Ann. Probab. 43 (2015), no. 5, 2647--2700. doi:10.1214/14-AOP946. https://projecteuclid.org/euclid.aop/1441792295


Export citation

References

  • [1] Ambrosio, L., Gigli, N. and Savaré, G. (2008). Gradient Flows in Metric Spaces and in the Space of Probability Measures, 2nd ed. Birkhäuser, Basel.
  • [2] Andersson, D. and Djehiche, B. (2011). A maximum principle for SDEs of mean-field type. Appl. Math. Optim. 63 341–356.
  • [3] Bensoussan, A., Sung, K. C. J., Yam, S. C. P. and Yung, S. P. (2011). Linear quadratic mean field games. Technical report.
  • [4] Buckdahn, R., Djehiche, B. and Li, J. (2011). A general stochastic maximum principle for SDEs of mean-field type. Appl. Math. Optim. 64 197–216.
  • [5] Buckdahn, R., Djehiche, B., Li, J. and Peng, S. (2009). Mean-field backward stochastic differential equations: A limit approach. Ann. Probab. 37 1524–1565.
  • [6] Cardaliaguet, P. (2012). Notes on mean field games. Notes from P. L. Lions’ lectures at the Collège de France. Available at https://www.ceremade.dauphine.fr/~cardalia/MFG100629.pdf.
  • [7] Carmona, R. and Delarue, F. (2014). The master equation for large population equilibriums. In Stochastic Analysis and Applications 2014 (B. Hambly, D. Crisan, T. Zariphopoulou and M. Reizakis, eds.) 77–128. Springer, Cham. MR3332710.
  • [8] Carmona, R. and Delarue, F. (2013). Mean field forward–backward stochastic differential equations. Electron. Commun. Probab. 18 1–15.
  • [9] Carmona, R. and Delarue, F. (2013). Probabilistic analysis of mean-field games. SIAM J. Control Optim. 51 2705–2734.
  • [10] Carmona, R., Delarue, F. and Lachapelle, A. (2013). Control of McKean–Vlasov dynamics versus mean field games. Math. Financ. Econ. 7 131–166.
  • [11] Delarue, F. (2002). On the existence and uniqueness of solutions to FBSDEs in a non-degenerate case. Stochastic Process. Appl. 99 209–286.
  • [12] Huang, M., Malhamé, R. P. and Caines, P. E. (2006). Large population stochastic dynamic games: Closed-loop McKean–Vlasov systems and the Nash certainty equivalence principle. Commun. Inf. Syst. 6 221–251.
  • [13] Jourdain, B., Méléard, S. and Woyczynski, W. A. (2008). Nonlinear SDEs driven by Lévy processes and related PDEs. ALEA Lat. Am. J. Probab. Math. Stat. 4 1–29.
  • [14] Lasry, J.-M. and Lions, P.-L. (2006). Jeux à champ moyen. I. Le cas stationnaire. C. R. Math. Acad. Sci. Paris 343 619–625.
  • [15] Lasry, J.-M. and Lions, P.-L. (2006). Jeux à champ moyen. II. Horizon fini et contrôle optimal. C. R. Math. Acad. Sci. Paris 343 679–684.
  • [16] Lasry, J.-M. and Lions, P.-L. (2007). Mean field games. Jpn. J. Math. 2 229–260.
  • [17] Lions, P. L. (2007/2008). Théorie des jeux à champs moyen et applications. Technical report.
  • [18] McKean, H. P. Jr. (1966). A class of Markov processes associated with nonlinear parabolic equations. Proc. Natl. Acad. Sci. USA 56 1907–1911.
  • [19] McKean, H. P. Jr. (1967). Propagation of chaos for a class of non-linear parabolic equations. In Stochastic Differential Equations (Lecture Series in Differential Equations, Session 7, Catholic Univ., 1967) 41–57. Air Force Office Sci. Res., Arlington, VA.
  • [20] Pardoux, É. and Peng, S. G. (1990). Adapted solution of a backward stochastic differential equation. Systems Control Lett. 14 55–61.
  • [21] Peng, S. and Wu, Z. (1999). Fully coupled forward–backward stochastic differential equations and applications to optimal control. SIAM J. Control Optim. 37 825–843.
  • [22] Rachev, S. T. and Rüschendorf, L. (1998). Mass Transportation Problems: Applications. Vol. II. Springer, New York.
  • [23] Sznitman, A.-S. (1991). Topics in propagation of chaos. In École D’Été de Probabilités de Saint-Flour XIX—1989 (D. L. Burkholder et al., eds.). Lecture Notes in Math. 1464 165–251. Springer, Berlin.
  • [24] Villani, C. (2009). Optimal Transport: Old and New. Grundlehren der Mathematischen Wissenschaften 338. Springer, Berlin.
  • [25] Wu, Z. and Yu, Z. (2014). Probabilistic interpretation for a system of quasilinear parabolic partial differential equation combined with algebra equations. Stochastic Process. Appl. 124 3921–3947.
  • [26] Yong, J. and Zhou, X. Y. (1999). Stochastic Controls: Hamiltonian Systems and HJB Equations. Applications of Mathematics (New York) 43. Springer, New York.