Open Access
dec 2005 Zero-sum continuous-time Markov games with unbounded transition and discounted payoff rates
Xianping Guo, Onésimo Hernández-Lerma
Author Affiliations +
Bernoulli 11(6): 1009-1029 (dec 2005). DOI: 10.3150/bj/1137421638


This paper is concerned with two-person zero-sum games for continuous-time Markov chains, with possibly unbounded payoff and transition rate functions, under the discounted payoff criterion. We give conditions under which the existence of the value of the game and a pair of optimal stationary strategies is ensured by using the optimality (or Shapley) equation. We prove the convergence of the value iteration scheme to the game's value and to a pair of optimal stationary strategies. Moreover, when the transition rates are bounded we further show that the convergence of value iteration is exponential. Our results are illustrated with a controlled queueing system with unbounded transition and reward rates.


Download Citation

Xianping Guo. Onésimo Hernández-Lerma. "Zero-sum continuous-time Markov games with unbounded transition and discounted payoff rates." Bernoulli 11 (6) 1009 - 1029, dec 2005.


Published: dec 2005
First available in Project Euclid: 16 January 2006

zbMATH: 1125.91016
MathSciNet: MR2188839
Digital Object Identifier: 10.3150/bj/1137421638

Keywords: controlled Q-process , discounted payoffs , value of the game , zero-sum Markov games

Rights: Copyright © 2005 Bernoulli Society for Mathematical Statistics and Probability

Vol.11 • No. 6 • dec 2005
Back to Top