August 2024 Learning to reflect: A unifying approach for data-driven stochastic control strategies
Sören Christensen, Claudia Strauch, Lukas Trottner
Author Affiliations +
Bernoulli 30(3): 2074-2101 (August 2024). DOI: 10.3150/23-BEJ1665

Abstract

Stochastic optimal control problems have a long tradition in applied probability, with the questions addressed being of high relevance in a multitude of fields. Even though theoretical solutions are well understood in many scenarios, their practicability suffers from the assumption of known dynamics of the underlying stochastic process, raising the statistical challenge of developing purely data-driven strategies. For the mathematically separated classes of continuous diffusion processes and Lévy processes, we show that developing efficient strategies for related singular stochastic control problems can essentially be reduced to finding rate-optimal estimators with respect to the sup-norm risk of objects associated to the invariant distribution of ergodic processes which determine the theoretical solution of the control problem. From a statistical perspective, we exploit the exponential β-mixing property as the common factor of both scenarios to drive the convergence analysis, indicating that relying on general stability properties of Markov processes is a sufficiently powerful and flexible approach to treat complex applications requiring statistical methods. We show moreover that in the Lévy case—even though per se jump processes are more difficult to handle both in statistics and control theory—a fully data-driven strategy with regret of significantly better order than in the diffusion case can be constructed utilizing spatial ergodicity of a path-time transformation of the Lévy process in form of its overshoots.

Acknowledgements

CS gratefully acknowledges financial support of Sapere Aude: DFF-Starting Grant 0165-00061B “Learning diffusion dynamics and strategies for optimal control”. LT was supported by Research Training Group “Statistical Modeling of Complex Systems” funded by the German Science Foundation. All three authors would like to thank two anonymous referees for the rigorous review and insightful suggestions that helped to improve the quality of the paper.

Citation

Download Citation

Sören Christensen. Claudia Strauch. Lukas Trottner. "Learning to reflect: A unifying approach for data-driven stochastic control strategies." Bernoulli 30 (3) 2074 - 2101, August 2024. https://doi.org/10.3150/23-BEJ1665

Information

Received: 1 September 2021; Published: August 2024
First available in Project Euclid: 14 May 2024

Digital Object Identifier: 10.3150/23-BEJ1665

Keywords: Data-driven singular control , Diffusion processes , exploration vs. exploitation , Lévy processes , nonparametric statistics , overshoots , reinforcement learning , sup-norm risk

JOURNAL ARTICLE
28 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

Vol.30 • No. 3 • August 2024
Back to Top