The Annals of Probability

Controlled Markov Chains

Harry Kesten and Frank Spitzer

Full-text: Open access

Abstract

We propose a control problem in which we minimize the expected hitting time of a fixed state in an arbitrary Markov chains with countable state space. A Markovian optimal strategy exists in all cases, and the value of this strategy is the unique solution of a nonlinear equation involving the transition function of the Markov chain.

Article information

Source
Ann. Probab., Volume 3, Number 1 (1975), 32-40.

Dates
First available in Project Euclid: 19 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aop/1176996445

Digital Object Identifier
doi:10.1214/aop/1176996445

Mathematical Reviews number (MathSciNet)
MR363616

Zentralblatt MATH identifier
0318.60070

JSTOR
links.jstor.org

Subjects
Primary: 60J05: Discrete-time Markov processes on general state spaces
Secondary: 60J20: Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) [See also 90B30, 91D10, 91D35, 91E40] 93E99: None of the above, but in this section

Keywords
Markov chains negative dynamic programming hitting times

Citation

Kesten, Harry; Spitzer, Frank. Controlled Markov Chains. Ann. Probab. 3 (1975), no. 1, 32--40. doi:10.1214/aop/1176996445. https://projecteuclid.org/euclid.aop/1176996445


Export citation