Open Access
2012 A Version of the Euler Equation in Discounted Markov Decision Processes
H. Cruz-Suárez, G. Zacarías-Espinoza, V. Vázquez-Guevara
J. Appl. Math. 2012: 1-16 (2012). DOI: 10.1155/2012/103698


This paper deals with Markov decision processes (MDPs) on Euclidean spaces with an infinite horizon. An approach to study this kind of MDPs is using the dynamic programming technique (DP). Then the optimal value function is characterized through the value iteration functions. The paper provides conditions that guarantee the convergence of maximizers of the value iteration functions to the optimal policy. Then, using the Euler equation and an envelope formula, the optimal solution of the optimal control problem is obtained. Finally, this theory is applied to a linear-quadratic control problem in order to find its optimal policy.


Download Citation

H. Cruz-Suárez. G. Zacarías-Espinoza. V. Vázquez-Guevara. "A Version of the Euler Equation in Discounted Markov Decision Processes." J. Appl. Math. 2012 1 - 16, 2012.


Published: 2012
First available in Project Euclid: 2 January 2013

zbMATH: 1272.49045
MathSciNet: MR2991589
Digital Object Identifier: 10.1155/2012/103698

Rights: Copyright © 2012 Hindawi

Vol.2012 • 2012
Back to Top