Stochastic continuum-armed bandits with additive models: Minimax regrets and adaptive algorithm

Abstract

We consider d-dimensional stochastic continuum-armed bandits with the expected reward function being additive β-Hölder with sparsity s for $0<\mathit{\beta }<\infty$ and $1\le \mathit{s}\le \mathit{d}$. The rate of convergence $\tilde{\mathit{O}}(\mathit{s}·{\mathit{T}}^{\frac{\mathit{\beta }+1}{2\mathit{\beta }+1}})$ for the minimax regret is established where T is the number of rounds. In particular, the minimax regret does not depend on d and is linear in s. A novel algorithm is proposed and is shown to be rate-optimal, up to a logarithmic factor of T.

The problem of adaptivity is also studied. A lower bound on the cost of adaptation to the smoothness is obtained and the result implies that adaptation for free is impossible in general without further structural assumptions. We then consider adaptive additive SCAB under an additional self-similarity assumption. An adaptive procedure is constructed and is shown to simultaneously achieve the minimax regret for a range of smoothness levels.