Journal of Applied Probability
- J. Appl. Probab.
- Volume 48, Number 4 (2011), 954-967.
Sensitivity analysis in Markov decision processes with uncertain reward parameters
Sequential decision problems can often be modeled as Markov decision processes. Classical solution approaches assume that the parameters of the model are known. However, model parameters are usually estimated and uncertain in practice. As a result, managers are often interested in how estimation errors affect the optimal solution. In this paper we illustrate how sensitivity analysis can be performed directly for a Markov decision process with uncertain reward parameters using the Bellman equations. In particular, we consider problems involving (i) a single stationary parameter, (ii) multiple stationary parameters, and (iii) multiple nonstationary parameters. We illustrate the applicability of this work through a capacitated stochastic lot-sizing problem.
J. Appl. Probab., Volume 48, Number 4 (2011), 954-967.
First available in Project Euclid: 16 December 2011
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Tan, Chin Hon; Hartman, Joseph C. Sensitivity analysis in Markov decision processes with uncertain reward parameters. J. Appl. Probab. 48 (2011), no. 4, 954--967. doi:10.1239/jap/1324046012. https://projecteuclid.org/euclid.jap/1324046012