Markov Strategies for Optimal Control Problems Indexed by a Partially Ordered Set

Abstract

We define the notion of a Markov strategy for the general optimal control problem where the index set is partially ordered. We prove that the supremum over all strategies is always attained by a Markov strategy if and only if the structure of the probability space is that of a Markov random field.