Translator Disclaimer
December 2012 Discounted continuous-time controlled Markov chains: convergence of control models
Tomás Prieto-Rumeau, Onésimo Hernández-Lerma
Author Affiliations +
J. Appl. Probab. 49(4): 1072-1090 (December 2012). DOI: 10.1239/jap/1354716658

Abstract

We are interested in continuous-time, denumerable state controlled Markov chains (CMCs), with compact Borel action sets, and possibly unbounded transition and reward rates, under the discounted reward optimality criterion. For such CMCs, we propose a definition of a sequence of control models {ℳn} converging to a given control model ℳ, which ensures that the discount optimal reward and policies of ℳn converge to those of ℳ. As an application, we propose a finite-state and finite-action truncation technique of the original control model ℳ, which is illustrated by approximating numerically the optimal reward and policy of a controlled population system with catastrophes. We study the corresponding convergence rates.

Citation

Download Citation

Tomás Prieto-Rumeau. Onésimo Hernández-Lerma. "Discounted continuous-time controlled Markov chains: convergence of control models." J. Appl. Probab. 49 (4) 1072 - 1090, December 2012. https://doi.org/10.1239/jap/1354716658

Information

Published: December 2012
First available in Project Euclid: 5 December 2012

zbMATH: 1255.90126
MathSciNet: MR3058989
Digital Object Identifier: 10.1239/jap/1354716658

Subjects:
Primary: 60J27 , 90C40

Keywords: approximation of control models , Continuous-time controlled Markov chain , discount optimality

Rights: Copyright © 2012 Applied Probability Trust

JOURNAL ARTICLE
19 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

SHARE
Vol.49 • No. 4 • December 2012
Back to Top