Open Access
Translator Disclaimer
February, 1975 Controlled Markov Chains
Harry Kesten, Frank Spitzer
Ann. Probab. 3(1): 32-40 (February, 1975). DOI: 10.1214/aop/1176996445

Abstract

We propose a control problem in which we minimize the expected hitting time of a fixed state in an arbitrary Markov chains with countable state space. A Markovian optimal strategy exists in all cases, and the value of this strategy is the unique solution of a nonlinear equation involving the transition function of the Markov chain.

Citation

Download Citation

Harry Kesten. Frank Spitzer. "Controlled Markov Chains." Ann. Probab. 3 (1) 32 - 40, February, 1975. https://doi.org/10.1214/aop/1176996445

Information

Published: February, 1975
First available in Project Euclid: 19 April 2007

zbMATH: 0318.60070
MathSciNet: MR363616
Digital Object Identifier: 10.1214/aop/1176996445

Subjects:
Primary: 60J05
Secondary: 60J20 , 93E99

Keywords: hitting times , Markov chains , negative dynamic programming

Rights: Copyright © 1975 Institute of Mathematical Statistics

JOURNAL ARTICLE
9 PAGES


SHARE
Vol.3 • No. 1 • February, 1975
Back to Top