Markov Decision Processes with a New Optimality Criterion: Small Interest Rates

Abstract

Finite state and action discrete time Markov decision processes with discounting are considered under the criterion of moment optimality. The case of small interest rates is studied, in particular the behavior of optimal policies as the interest rate approaches zero. Laurent expansions in the interest rate are developed for all moments of return for stationary policies, and a proof is given that there is a stationary policy which is moment optimal for all sufficiently small interest rates.