Open Access
August 2017 Constrained total undiscounted continuous-time Markov decision processes
Xianping Guo, Yi Zhang
Bernoulli 23(3): 1694-1736 (August 2017). DOI: 10.3150/15-BEJ793


The present paper considers the constrained optimal control problem with total undiscounted criteria for a continuous-time Markov decision process (CTMDP) in Borel state and action spaces. The cost rates are nonnegative. Under the standard compactness and continuity conditions, we show the existence of an optimal stationary policy out of the class of general nonstationary ones. In the process, we justify the reduction of the CTMDP model to a discrete-time Markov decision process (DTMDP) model based on the studies of the undiscounted occupancy and occupation measures. We allow that the controlled process is not necessarily absorbing, and the transition rates are not necessarily separated from zero, and can be arbitrarily unbounded; these features count for the main technical difficulties in studying undiscounted CTMDP models.


Download Citation

Xianping Guo. Yi Zhang. "Constrained total undiscounted continuous-time Markov decision processes." Bernoulli 23 (3) 1694 - 1736, August 2017.


Received: 1 April 2015; Revised: 1 September 2015; Published: August 2017
First available in Project Euclid: 17 March 2017

zbMATH: 06714316
MathSciNet: MR3624875
Digital Object Identifier: 10.3150/15-BEJ793

Keywords: constrained optimality , continuous-time Markov decision processes , total undiscounted criteria

Rights: Copyright © 2017 Bernoulli Society for Mathematical Statistics and Probability

Vol.23 • No. 3 • August 2017
Back to Top