Source: J. Appl. Probab. Volume 46, Number 4
(2009), 1157-1183.
This work is concerned with the existence of an optimal control strategy
for the long-run average continuous control problem of
piecewise-deterministic Markov processes (PDMPs). In Costa and Dufour
(2008), sufficient conditions were derived to ensure the existence of an
optimal control by using the vanishing discount approach. These conditions
were mainly expressed in terms of the relative difference of the
α-discount
value functions. The main goal of this paper is to
derive tractable conditions directly related to the primitive data of the
PDMP to ensure the existence of an optimal control. The present work can
be seen as a continuation of the results derived in Costa and Dufour
(2008). Our main assumptions are written in terms of some
integro-differential inequalities related to the so-called expected growth
condition, and geometric convergence of the post-jump location kernel
associated to the PDMP. An example based on the capacity expansion problem
is presented, illustrating the possible applications of the results
developed in the paper.
Full-text: Access denied (no subscription
detected)
We're sorry, but we are unable to provide
you with the full text of this article because we are not able to identify
you as a subscriber.
If you have a personal subscription to
this journal, then please login. If you are already logged in, then you
may need to update your profile to register your subscription.
Read more about accessing full-text
References
Costa, O. L. V. and Dufour, F. (2007). Relaxed long-run average continuous control of piecewise deterministic Markov processes. In Proc. European Control Conf., Kos, Greece, 5052--5059.
Costa, O. L. V. and Dufour, F. (2008). Average continuous control of piecewise deterministic Markov processes. Submitted. Available at http://arxiv.org/abs/0809.0477.
Costa, O. L. V. and Dufour, F. (2008). Stability and ergodicity of piecewise deterministic Markov processes. SIAM J. Control Optimization 47, 1053--1077.
Davis, M. H. A. (1984). Piecewise-deterministic Markov processes: a general class of non-diffusion stochastic models. J. R. Statist. Soc. B 46, 353--388.
Mathematical Reviews (MathSciNet):
MR790622
Davis, M. H. A. (1993). Markov Models and Optimization. Chapman and Hall, London.
Davis, M. H. A., Dempster, M. A. H., Sethi, S. P. and Vermes, D. (1987). Optimal capacity expansion under uncertainty. Adv. Appl. Prob. 19, 156--176.
Mathematical Reviews (MathSciNet):
MR876535
Dufour, F. and Costa, O. L. V. (1999). Stability of piecewise-deterministic Markov processes. SIAM J. Control Optimization 37, 1483--1502.
Guo, X. and Rieder, U. (2006). Average optimality for continuous-time Markov decision processes in Polish spaces. Ann. Appl. Prob. 16, 730--756.
Guo, X. and Zhu, Q. (2006). Average optimality for Markov decision processes in Borel spaces: a new condition and approach. J. Appl. Prob. 43, 318--334.
Hernández-Lerma, O. and Lasserre, J. B. (1996). Discrete-Time Markov Control Processes (Appl. Math. 30). Springer, New York.
Hernández-Lerma, O. and Lasserre, J. B. (1999). Further Topics on Discrete-Time Markov Control Processes (Appl. Math. 42). Springer, New York.
Luss, H. (1982). Operations research and capacity expansion problems: a survey. Operat. Res. 30, 907--947.
Meyn, S. and Tweedie, R. (1992). Stability of Markovian processes. I. Criteria for discrete-time chains. Adv. Appl. Prob. 24, 542--574.
Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, Berlin.
Widder, D. V. (1941). The Laplace Transform (Princeton Math. Ser. v. 6). Princeton University Press.
Mathematical Reviews (MathSciNet):
MR5923
Zhu, Q. (2008). Average optimality for continuous-time Markov decision processes with a policy iteration approach. J. Math. Anal. Appl. 339, 691--704.
