An Example in which Stationary Strategies are not Adequate
Lester E. Dubins, William D. Sudderth · The Annals of Probability · 1975
An example is given of a gambling problem such that, on the one hand, for every initial state, there is a strategy which brings one to the goal with arbitrarily high probability and, on the other hand, for some initial state, every stationary strategy reaches the goal with probability zero.