A Countable Policy Set for Sequential Decision Problems

Abstract

In denumerable state, denumerable action sequential decision problems in which the reward function has uniformly bounded 2nd moment, the optimal reward for the decisionmaker who restricts himself to the countable set of stationary policies consisting of those which choose some arbitrary action at all but a finite number of states will be the same as the optimal reward for the decisionmaker who optimizes over all stationary policies. Under some further restriction, he can do almost as well simply by solving a large finite state truncation of the original problem.