The Annals of Applied Probability

Dynamic approaches for some time-inconsistent optimization problems

Chandrasekhar Karnam, Jin Ma, and Jianfeng Zhang

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


In this paper, we investigate possible approaches to study general time-inconsistent optimization problems without assuming the existence of optimal strategy. This leads immediately to the need to refine the concept of time consistency as well as any method that is based on Pontryagin’s maximum principle. The fundamental obstacle is the dilemma of having to invoke the Dynamic Programming Principle (DPP) in a time-inconsistent setting, which is contradictory in nature. The main contribution of this work is the introduction of the idea of the “dynamic utility” under which the original time-inconsistent problem (under the fixed utility) becomes a time-consistent one. As a benchmark model, we shall consider a stochastic controlled problem with multidimensional backward SDE dynamics, which covers many existing time-inconsistent problems in the literature as special cases; and we argue that the time inconsistency is essentially equivalent to the lack of comparison principle. We shall propose three approaches aiming at reviving the DPP in this setting: the duality approach, the dynamic utility approach and the master equation approach. Unlike the game approach in many existing works in continuous time models, all our approaches produce the same value as the original static problem.

Article information

Ann. Appl. Probab., Volume 27, Number 6 (2017), 3435-3477.

Received: April 2016
Revised: January 2017
First available in Project Euclid: 15 December 2017

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 49L20: Dynamic programming method 60H10: Stochastic ordinary differential equations [See also 34F05] 91C99: None of the above, but in this section 91G80: Financial applications of other theories (stochastic control, calculus of variations, PDE, SPDE, dynamical systems) 35R15: Partial differential equations on infinite-dimensional (e.g. function) spaces (= PDE in infinitely many variables) [See also 46Gxx, 58D25]

Time inconsistency dynamic programming principle stochastic maximum principle comparison principle duality dynamic utility master equation path derivative


Karnam, Chandrasekhar; Ma, Jin; Zhang, Jianfeng. Dynamic approaches for some time-inconsistent optimization problems. Ann. Appl. Probab. 27 (2017), no. 6, 3435--3477. doi:10.1214/17-AAP1284.

Export citation


  • [1] Aubin, J.-P. and Frankowska, H. (2009). Set-Valued Analysis. Birkhäuser, Boston, MA.
  • [2] Bjork, T. and Murgoci, A. (2010). A general theory of markovian time inconsistent stochastic control problems. Available at
  • [3] Björk, T., Murgoci, A. and Zhou, X. Y. (2014). Mean-variance portfolio optimization with state-dependent risk aversion. Math. Finance 24 1–24.
  • [4] Bouchard, B., Elie, R. and Touzi, N. (2009/10). Stochastic target problems with controlled loss. SIAM J. Control Optim. 48 3123–3150.
  • [5] Cardaliaguet, P., Delarue, F., Lasry, J. M. and Lions, P. L. (2015). The master equation and the convergence problem in mean field games. Preprint. Available at arXiv:1509.02505.
  • [6] Cohen, S. and Elliott, R. (2009). Time consistency and moving horizons for risk measures. Preprint. Available at arXiv:0912.1396.
  • [7] Cont, R. and Fournié, D.-A. (2013). Functional Itô calculus and stochastic integral representation of martingales. Ann. Probab. 41 109–133.
  • [8] Cui, X., Li, D., Wang, S. and Zhu, S. (2012). Better than dynamic mean-variance: Time inconsistency and free cash flow stream. Math. Finance 22 346–378.
  • [9] Cvitanić, J. and Zhang, J. (2013). Contract Theory in Continuous-Time Models. Springer, Heidelberg.
  • [10] Dupire, B. (2009). Functional Itô calculus. Preprint. Available at
  • [11] Ekeland, I. and Lazrak, A. (2010). The golden rule when preferences are time inconsistent. Math. Financ. Econ. 4 29–55.
  • [12] Ekren, I., Keller, C., Touzi, N. and Zhang, J. (2014). On viscosity solutions of path dependent PDEs. Ann. Probab. 42 204–236.
  • [13] Ekren, I., Touzi, N. and Zhang, J. (2016). Viscosity solutions of fully nonlinear parabolic path dependent PDEs: Part I. Ann. Probab. 44 1212–1253.
  • [14] Ekren, I., Touzi, N. and Zhang, J. (2016). Viscosity solutions of fully nonlinear parabolic path dependent PDEs: Part II. Ann. Probab. 44 2507–2553.
  • [15] Ekren, I., Touzi, N. and Zhang, J. (2016). Viscosity solutions of fully nonlinear parabolic path dependent PDEs: Part I. Ann. Probab. 44 1212–1253.
  • [16] Ekren, I. and Zhang, J. (2016). Pseudo-Markovian viscosity solutions of fully nonlinear degenerate PPDEs. Probab. Uncertain. Quant. Risk 1 Paper No. 6, 34.
  • [17] El Karoui, N. and Mrad, M. (2013). An exact connection between two solvable SDEs and a nonlinear utility stochastic PDE. SIAM J. Financial Math. 4 697–736.
  • [18] Feinstein, Z. and Rudloff, B. (2013). Time consistency of dynamic risk measures in markets with transaction costs. Quant. Finance 13 1473–1489.
  • [19] Feinstein, Z. and Rudloff, B. (2016). Time consistency for scalar multivariate risk measures. Working paper.
  • [20] Hu, Y., Jin, H. and Zhou, X. Y. (2012). Time-inconsistent stochastic linear-quadratic control. SIAM J. Control Optim. 50 1548–1572.
  • [21] Hu, Y. and Peng, S. (2006). On the comparison theorem for multidimensional BSDEs. C. R. Math. Acad. Sci. Paris 343 135–140.
  • [22] Kahneman, D. and Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica 47 263–292.
  • [23] Kahneman, D. and Tversky, A. (1992). Advances in prospect theory: Cumulative representation of uncertainty. J. Risk and Uncertainty 5 297–323.
  • [24] Keller, C. and Zhang, J. (2016). Pathwise Itô calculus for rough paths and rough PDEs with path dependent coefficients. Stochastic Process. Appl. 126 735–766.
  • [25] Kydland, F. and Prescott, E. (1977). Rules rather than discretion: The inconsistency of optimal plans. J. Polit. Econ. 85 473–492.
  • [26] Ma, J. and Yong, J. (1995). Solvability of forward–backward SDEs and the nodal set of Hamilton–Jacobi–Bellman equations. Chin. Ann. Math. Ser. B 16 279–298. [A Chinese summary appears in Chinese Ann. Math. Ser. A 16 (1995), no. 4, 532.]
  • [27] Miller, C. W. (2017). Nonlinear PDE approach to time-inconsistent optimal stopping. SIAM J. Control Optim. 55 557–573.
  • [28] Musiela, M. and Zariphopoulou, T. (2007). Investment and valuation under backward and forward dynamic exponential utilities in a stochastic factor model. In Advances in Mathematical Finance. Appl. Numer. Harmon. Anal. 303–334. Birkhäuser, Boston, MA.
  • [29] Musiela, M. and Zariphopoulou, T. (2010). Stochastic partial differential equations and portfolio choice. In Contemporary Quantitative Finance 195–216. Springer, Berlin.
  • [30] Ren, Z. and Tan, X. (2017). On the convergence of monotone schemes for path-dependent PDEs. Stochastic Process. Appl. 127 1738–1762.
  • [31] Soner, H. M. and Touzi, N. (2002). Dynamic programming for stochastic target problems and geometric flows. J. Eur. Math. Soc. (JEMS) 4 201–236.
  • [32] Strotz, R. H. (1955). Myopia and inconsistency in dynamic utility maximization. Rev. Econ. Stud. 23 165–180.
  • [33] Xu, Z. Q. and Zhou, X. Y. (2013). Optimal stopping under probability distortion. Ann. Appl. Probab. 23 251–282.
  • [34] Yong, J. (2012). Time-inconsistent optimal control problems and the equilibrium HJB equation. Math. Control Relat. Fields 2 271–329.
  • [35] Zhang, J. and Zhuo, J. (2014). Monotone schemes for fully nonlinear parabolic path dependent PDEs. J. Financ. Eng. 1 1450005. (23 pages).
  • [36] Zhou, X. Y. (2010). Mathematicalising behavioural finance. In Proceedings of the International Congress of Mathematicians. Volume IV 3185–3209. Hindustan Book Agency, New Delhi.