The Annals of Statistics

Markov Decision Processes with a New Optimality Criterion: Continuous Time

Stratton C. Jaquette

Full-text: Open access

Abstract

Standard finite state and action continuous time Markov decision processes with discounting are studied using a new optimality criterion called moment optimality. A policy is moment optimal if it lexicographically maximizes the sequence of signed moments of total discounted return with a positive (negative) sign if the moment is odd (even). It is shown constructively that a stationary policy is moment optimal among the class of piecewise constant policies by examining the negative of the Laplace transform of the total return random variable and its Taylor series expansion.

Article information

Source
Ann. Statist., Volume 3, Number 2 (1975), 547-553.

Dates
First available in Project Euclid: 12 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aos/1176343087

Digital Object Identifier
doi:10.1214/aos/1176343087

Mathematical Reviews number (MathSciNet)
MR363493

Zentralblatt MATH identifier
0321.90051

JSTOR
links.jstor.org

Subjects
Primary: 90B99: None of the above, but in this section
Secondary: 60J25: Continuous-time Markov processes on general state spaces 93E20: Optimal stochastic control 90C40: Markov and semi-Markov decision processes 90B99: None of the above, but in this section

Keywords
Dynamic programming Markov decision processes optimality criterion moments of return

Citation

Jaquette, Stratton C. Markov Decision Processes with a New Optimality Criterion: Continuous Time. Ann. Statist. 3 (1975), no. 2, 547--553. doi:10.1214/aos/1176343087. https://projecteuclid.org/euclid.aos/1176343087


Export citation