An example is given of a gambling problem such that, on the one hand, for every initial state, there is a strategy which brings one to the goal with arbitrarily high probability and, on the other hand, for some initial state, every stationary strategy reaches the goal with probability zero.
"An Example in which Stationary Strategies are not Adequate." Ann. Probab. 3 (4) 722 - 725, August, 1975. https://doi.org/10.1214/aop/1176996312