If a set of states is given in a problem of dynamic programming in which each state can be observed only partially, the given model is generally transformed into a new model with completely observed states. In this article a method is introduced with which Markov models of dynamic programming can be transformed and which preserves the Markov property. The method applies to relatively general sets of states.
"Incomplete Information in Markovian Decision Models." Ann. Statist. 2 (6) 1327 - 1334, November, 1974. https://doi.org/10.1214/aos/1176342886