A characterization of the optimal risk-sensitive average cost in finite controlled Markov chains
Rolando Cavazos-Cadena and Daniel Hernández-Hernández
Source: Ann. Appl. Probab. Volume 15, Number 1A
(2005), 175-212.
Abstract
This work concerns controlled Markov chains with finite state and action spaces. The transition law satisfies the simultaneous Doeblin condition, and the performance of a control policy is measured by the (long-run) risk-sensitive average cost criterion associated to a positive, but otherwise arbitrary, risk sensitivity coefficient. Within this context, the optimal risk-sensitive average cost is characterized via a minimization problem in a finite-dimensional Euclidean space.
First Page:
Show
Hide
Keywords: Decreasing function along trajectories; stopping time; nearly optimal policies; Hölder’s inequality; simultaneous Doeblin condition; recurrent state
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.aoap/1106922326
Digital Object Identifier: doi:10.1214/105051604000000585
Mathematical Reviews number (MathSciNet): MR2115041
Zentralblatt MATH identifier: 02162825
References
Bielecki, T., Hernández-Hernández, D. and Pliska, S. (1999). Risk sensitive control of finite state Markov chains in discrete time, with applications to portfolio management. Math. Methods Oper. Res. 50 167--188.
Mathematical Reviews (MathSciNet): MR1732397
Digital Object Identifier: doi:10.1007/s001860050094
Bielecki, T. and Pliska, S. (1999). Risk sensitive dynamic asset management. Appl. Math. Optim. 39 337--360.
Mathematical Reviews (MathSciNet): MR1675114
Digital Object Identifier: doi:10.1007/s002459900110
Zentralblatt MATH: 0984.91047
Cavazos-Cadena, R. and Fernández-Gaucherand, E. (1999). Controlled Markov chains with risk-sensitive criteria: Average cost, optimality equations, and optimal solutions. Math. Methods Oper. Res. 49 299--324.
Mathematical Reviews (MathSciNet): MR1687362
Cavazos-Cadena, R. and Fernández-Gaucherand, E. (2001). Risk-sensitive optimal control in communicating average Markov decision chains. In Modeling Uncertainty: An Examination of Stochastic Theory, Methods, and Applications (M. Dror, P. L'Ecuyer and F. Szydarovszky, eds.) 515--553. Kluwer Academic, Dordrecht.
Mathematical Reviews (MathSciNet): MR1893292
Digital Object Identifier: doi:10.1007/0-306-48102-2_22
Cavazos-Cadena, R. and Hernández-Hernández, D. (2002). Solution to the risk-sensitive average optimality equation in communicating Markov decision chains with finite state space: An alternative approach. Math. Methods Oper. Res. 56 473--479.
Mathematical Reviews (MathSciNet): MR1953028
Digital Object Identifier: doi:10.1007/s001860200229
Cavazos-Cadena, R. and Hernández-Hernández, D. (2004). A characterization of exponential functionals in finite Markov chains. Math. Methods Oper. Res. To appear.
Mathematical Reviews (MathSciNet): MR2106091
Digital Object Identifier: doi:10.1007/s001860400373
Dembo, A. and Zeitouni, O. (1993). Large Deviations Techniques and Applications. Jones and Bartlett, Boston.
Mathematical Reviews (MathSciNet): MR1202429
Zentralblatt MATH: 0793.60030
Di Masi, G. B. and Stettner, L. (1999). Risk-sensitive control of discrete-time Markov processes with infinite horizon. SIAM J. Control Optim. 38 61--78.
Mathematical Reviews (MathSciNet): MR1740607
Digital Object Identifier: doi:10.1137/S0363012997320614
Zentralblatt MATH: 0946.93043
Donsker, M. D. and Varadhan, S. R. (1975). Asymptotic evaluation of certain Markov process expectations for large time, I, II, III. Comm. Pure Appl. Math. 28 1--45, 279--301; 29 389--461.
Mathematical Reviews (MathSciNet): MR386024
Dynkin, E. B. and Yushkevich, A. A. (1979). Controlled Markov Processes. Springer, Berlin.
Mathematical Reviews (MathSciNet): MR554083
Fleming, W. H. and Hernández-Hernández, D. (1997). Risk sensitive control of finite state machines on an infinite horizon I. SIAM J. Control Optim. 35 1790--1810.
Mathematical Reviews (MathSciNet): MR1466928
Digital Object Identifier: doi:10.1137/S0363012995291622
Zentralblatt MATH: 0891.93085
Fleming, W. H. and Sheu, S. (1999). Optimal long term growth rate of expected utility of wealth. Ann. Appl. Probab. 9 871--903.
Mathematical Reviews (MathSciNet): MR1722286
Digital Object Identifier: doi:10.1214/aoap/1029962817
Project Euclid: euclid.aoap/1029962817
Zentralblatt MATH: 0962.91036
Gale, D. (1968). A mathematical theory of optimal economic development. Bull. Amer. Math. Soc. 74 207--223.
Mathematical Reviews (MathSciNet): MR221835
Digital Object Identifier: doi:10.1090/S0002-9904-1968-11891-9
Hernández-Hernández, D. and Marcus, S. I. (1996). Risk sensitive control of Markov processes in countable state space. Systems Control Lett. 29 147--155. [Corrigendum (1998) 34 105--106.]
Mathematical Reviews (MathSciNet): MR1422212
Hernández-Lerma, O. (1988). Adaptive Markov Control Processes. Springer, New York.
Mathematical Reviews (MathSciNet): MR995463
Hordjik, A. (1974). Dynamic Programming and Potential Theory. Math. Centrum, Amsterdam.
Howard, A. R. and Matheson, J. E. (1972). Risk-sensitive Markov decision processes. Managment Sciences 18 356--369.
Mathematical Reviews (MathSciNet): MR292497
Puterman M. L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York.
Mathematical Reviews (MathSciNet): MR1270015
Zentralblatt MATH: 0829.90134
Thomas L. C. (1980). Connectedness conditions for denumerable state Markov decision processes. In Recent Advances in Markov Decision Processes (R. Hartles, L. C. Thomas and D. J. White, eds.) 181--204. Academic Press, New York.
The Annals of Applied Probability
