Stochastic online convex optimization. Application to probabilistic time series forecasting

Abstract

In this paper, we propose a general framework for stochastic online convex optimization that allows for achieving fast-rate stochastic regret bounds. Specifically, we demonstrate that certain algorithms, including online Newton steps and a scale-free variant of Bernstein online aggregation, achieve the best-known rates in unbounded stochastic settings. To illustrate the usefulness of our approach, we apply it to calibrating parametric probabilistic forecasters of non-stationary sub-Gaussian time series. Importantly, our fast-rate stochastic regret bounds are valid at any time, providing a flexible and robust performance metric for sequential algorithms. Our proofs rely on combining self-bounded and Poissonian inequalities for martingales and sub-Gaussian random variables, respectively, under a stochastic exp-concavity assumption.