Open Access
Translator Disclaimer
July 1996 Occupation measures for controlled Markov processes: characterization and optimality
Abhay G. Bhatt, Vivek S. Borkar
Ann. Probab. 24(3): 1531-1562 (July 1996). DOI: 10.1214/aop/1065725192


For controlled Markov processes taking values in a Polish space, control problems with ergodic cost, infinite-horizon discounted cost and finite-horizon cost are studied. Each is posed as a convex optimization problem wherein one tries to minimize a linear functional on a closed convex set of appropriately defined occupation measures for the problem. These are characterized as solutions of a linear equation asssociated with the problem. This characterization is used to establish the existence of optimal Markov controls. The dual convex optimization problem is also studied.


Download Citation

Abhay G. Bhatt. Vivek S. Borkar. "Occupation measures for controlled Markov processes: characterization and optimality." Ann. Probab. 24 (3) 1531 - 1562, July 1996.


Published: July 1996
First available in Project Euclid: 9 October 2003

zbMATH: 0863.93086
MathSciNet: MR1411505
Digital Object Identifier: 10.1214/aop/1065725192

Primary: 93E20
Secondary: 60J25

Keywords: Controlled Markov processes , infinite-dimensional linear programming , occupation measures , optimal control

Rights: Copyright © 1996 Institute of Mathematical Statistics


Vol.24 • No. 3 • July 1996
Back to Top