Stay-with-a-Winner Rule for Dependent Bernoulli Bandits

Abstract

The $k$-armed bandit problem on the Bernoulli dependent arms is discussed. Order relations on the prior distributions of the Bernoulli parameters using moments of the posterior are used to prove a monotonicity property of the value function. When $k = 2$, a stay-with-a-winner rule is derived for negatively correlated arms and for a certain class of positively correlated arms. These results are extensions of those given in Berry and Fristedt for independent Bernoulli arms. They also generalize the results of Benzing, Hinderer and Kolonko and Kolonko and Benzing.