Open Access
Translator Disclaimer
May, 1993 Optimal Control and Replacement with State-Dependent Failure Rate: Dynamic Programming
Arthur C. Heinricher, Richard H. Stockbridge
Ann. Appl. Probab. 3(2): 364-379 (May, 1993). DOI: 10.1214/aoap/1177005429


A class of stochastic control problems where the payoff depends on the running maximum of a diffusion process is described. The controller must make two kinds of decision: first, he must choose a work rate (this decision determines the rate of profit as well as the proximity of failure), and second, he must decide when to replace a deteriorated system with a new one. Preventive replacement is a realistic option if the cost for replacement after failure is larger than the cost of a preventive replacement. We focus on the profit and replacement cost for a single work cycle and solve the problem in two stages. First, the optimal feedback control (work rate) is determined by maximizing the payoff during a single excursion of a controlled diffusion away from the running maximum. This step involves the solution of the Hamilton-Jacobi-Bellman (HJB) partial differential equation. The second step is to determine the optimal replacement set. The assumption that failure occurs only on the set where the state is increasing implies that replacement is optimal only on this set. This leads to a simple formula for the optimal replacement level in terms of the value function.


Download Citation

Arthur C. Heinricher. Richard H. Stockbridge. "Optimal Control and Replacement with State-Dependent Failure Rate: Dynamic Programming." Ann. Appl. Probab. 3 (2) 364 - 379, May, 1993.


Published: May, 1993
First available in Project Euclid: 19 April 2007

zbMATH: 0781.93097
MathSciNet: MR1221157
Digital Object Identifier: 10.1214/aoap/1177005429

Primary: 49B60
Secondary: 49C20 , 93E20

Keywords: controlled diffusion , dynamic programming , optimal replacement , running maximum , state dependent failure , stochastic wear models

Rights: Copyright © 1993 Institute of Mathematical Statistics


Vol.3 • No. 2 • May, 1993
Back to Top