We analyze Gittins' Markovian model, as generalized by Varaiya, Walrand and Buyukkoc, in discrete and continuous time. The approach resembles Weber's modification of Whittle's, within the framework of both multi-parameter processes and excursion theory. It is shown that index-priority strategies are optimal, in concert with all the special cases that have been treated previously.
"Multi-armed bandits in discrete and continuous time." Ann. Appl. Probab. 8 (4) 1270 - 1290, November 1998. https://doi.org/10.1214/aoap/1028903380