Open Access
Translator Disclaimer
May, 1993 Optimal Control and Replacement with State-Dependent Failure Rate: An Invariant Measure Approach
Arthur C. Heinricher, Richard H. Stockbridge
Ann. Appl. Probab. 3(2): 380-402 (May, 1993). DOI: 10.1214/aoap/1177005430


Stochastic control problems in which the payoff depends on the running maximum of a diffusion process are considered. Such processes provide appealing models for physical processes that evolve in a continuous and increasing manner and fail at a random time. The controller must make two decisions: first, she must choose how fast to work (this decision determines the rate of profit as well as the proximity of failure), and second, she must decide when to replace a deteriorated system with a new one. Preventive replacement becomes an important option when the cost for replacement after a failure is larger than the cost of a preventive replacement. Single-cycle and long-term average criteria are used to evaluate the control and replacement decisions. We model the process via a martingale problem formulation. This enables the long-term average control problem to be rephrased as an LP over the invariant measures of the process. We identify the invariant measures corresponding to each control and replacement decision and determine the optimal solution using an iterative scheme.


Download Citation

Arthur C. Heinricher. Richard H. Stockbridge. "Optimal Control and Replacement with State-Dependent Failure Rate: An Invariant Measure Approach." Ann. Appl. Probab. 3 (2) 380 - 402, May, 1993.


Published: May, 1993
First available in Project Euclid: 19 April 2007

zbMATH: 0781.93098
MathSciNet: MR1221158
Digital Object Identifier: 10.1214/aoap/1177005430

Primary: 49A60
Secondary: 60G35 , 60G44 , 93E20

Keywords: constrained martingale problems , controlled diffusion , Controlled martingale problems , Invariant measures , optimal replacement , running maximum , state dependent failure , stochastic wear models

Rights: Copyright © 1993 Institute of Mathematical Statistics


Vol.3 • No. 2 • May, 1993
Back to Top