A recent result of Blackwell states that in a positive dynamic programming problem with countable state space, if there is an optimal policy, then there is a stationary optimal policy. We extend this result by allowing the state space to be Borel and by proving that if there is an optimal policy, then for any probability measure $\mu$ on the state space there is a stationary policy which is optimal on a set of $\mu$ measure 1.
"On Stationary Policies--The General Case." Ann. Statist. 2 (1) 219 - 222, January, 1974. https://doi.org/10.1214/aos/1176342629