Determining the Optimal Control of Singular Stochastic Processes Using Linear Programming

Abstract

This paper examines the numerical implementation of a linear programming (LP) formulation of stochastic control problems involving singular stochastic processes. The decision maker has the ability to influence a diffusion process through the selection of its drift rate (a control that acts absolutely continuously in time) and may also decide to instantaneously move the process to some other level (a singular control). The first goal of the paper is to show that linear programming provides a viable approach to solving singular control problems. A second goal is the determination of the absolutely continuous control from the LP results and is intimately tied to the particular numerical implementation. The original stochastic control problem is equivalent to an infinite-dimensional linear program in which the variables are measures on appropriate bounded regions. The implementation method replaces the LP formulation involving measures by one involving the moments of the measures. This moment approach does not directly provide the optimal control in feedback form of the current state. The second goal of this paper is to show that the feedback form of the optimal control can be obtained using sensitivity analysis.