Open Access
September 2020 Markov decision processes with dynamic transition probabilities: An analysis of shooting strategies in basketball
Nathan Sandholtz, Luke Bornn
Ann. Appl. Stat. 14(3): 1122-1145 (September 2020). DOI: 10.1214/20-AOAS1348


In this paper we model basketball plays as episodes from team-specific nonstationary Markov decision processes (MDPs) with shot clock dependent transition probabilities. Bayesian hierarchical models are employed in the modeling and parametrization of the transition probabilities to borrow strength across players and through time. To enable computational feasibility, we combine lineup-specific MDPs into team-average MDPs using a novel transition weighting scheme. Specifically, we derive the dynamics of the team-average process such that the expected transition count for an arbitrary state-pair is equal to the weighted sum of the expected counts of the separate lineup-specific MDPs.

We then utilize these nonstationary MDPs in the creation of a basketball play simulator with uncertainty propagated via posterior samples of the model components. After calibration, we simulate seasons both on-policy and under altered policies and explore the net changes in efficiency and production under the alternate policies. Additionally, we discuss the game-theoretic ramifications of testing alternative decision policies.


Download Citation

Nathan Sandholtz. Luke Bornn. "Markov decision processes with dynamic transition probabilities: An analysis of shooting strategies in basketball." Ann. Appl. Stat. 14 (3) 1122 - 1145, September 2020.


Received: 1 December 2018; Revised: 1 March 2020; Published: September 2020
First available in Project Euclid: 18 September 2020

MathSciNet: MR4152126
Digital Object Identifier: 10.1214/20-AOAS1348

Keywords: Basketball , Bayesian hierarchical model , Markov decision process , optical tracking data , simulation

Rights: Copyright © 2020 Institute of Mathematical Statistics

Vol.14 • No. 3 • September 2020
Back to Top