June 2023 A dynamic analytic method for risk-aware controlled martingale problems
Jukka Isohätälä, William B. Haskell
Author Affiliations +
Ann. Appl. Probab. 33(3): 1661-1700 (June 2023). DOI: 10.1214/22-AAP1794


We present a new, tractable method for solving risk-aware problems over finite and infinite, discounted time-horizons where the dynamics of the controlled process are described using the martingale method. Supposing general Polish state and action spaces, and using the martingale characterization, we state a risk-aware dynamic optimal control problem of minimizing risk of costs described by a generic risk function. From this, we construct an alternative formulation of the optimization problem that takes the form of a nonlinear programming problem, constrained by the dynamic, that is, time-dependent and linear Kolmogorov forward equation describing the time-dependent distribution of the state and running costs. This formulation is similar to the convex analytic method, in that the control problem is recast into a form where the objective is optimized over distributions representing the state space visitation frequencies. However, in our approach, the distributions are dynamic and also encode the cost distribution. As our main results, we prove the equivalence of the original martingale and dynamic analytic problems, in the sense that both have the same optimal values, and that the solution of either problem yields a solution of the other. Moreover, we find an optimal control process can be taken to be Markov in the controlled process state, running costs, and time. We further show that under additional assumptions the optimal value is attained. An example numeric problem is presented and solved.

Funding Statement

The authors would gratefully like to acknowledge support for this research from the Singapore Ministry of Education, under Tier 2 project MOE-2015-T2-2148.


Download Citation

Jukka Isohätälä. William B. Haskell. "A dynamic analytic method for risk-aware controlled martingale problems." Ann. Appl. Probab. 33 (3) 1661 - 1700, June 2023. https://doi.org/10.1214/22-AAP1794


Received: 1 June 2020; Revised: 1 October 2021; Published: June 2023
First available in Project Euclid: 2 May 2023

MathSciNet: MR4583655
zbMATH: 1518.93154
Digital Object Identifier: 10.1214/22-AAP1794

Primary: 60J25 , 93E20
Secondary: 60J35 , 90C30

Keywords: forward equation , Martingale problems , nonlinear programming , optimal control , Stochastic processes

Rights: Copyright © 2023 Institute of Mathematical Statistics


This article is only available to subscribers.
It is not available for individual sale.

Vol.33 • No. 3 • June 2023
Back to Top