The Annals of Applied Probability

Optimal stopping and free boundary characterizations for some Brownian control problems

Amarjit Budhiraja and Kevin Ross

Full-text: Open access


A singular stochastic control problem with state constraints in two-dimensions is studied. We show that the value function is C1 and its directional derivatives are the value functions of certain optimal stopping problems. Guided by the optimal stopping problem, we then introduce the associated no-action region and the free boundary and show that, under appropriate conditions, an optimally controlled process is a Brownian motion in the no-action region with reflection at the free boundary. This proves a conjecture of Martins, Shreve and Soner [SIAM J. Control Optim. 34 (1996) 2133–2171] on the form of an optimal control for this class of singular control problems. An important issue in our analysis is that the running cost is Lipschitz but not C1. This lack of smoothness is one of the key obstacles in establishing regularity of the free boundary and of the value function. We show that the free boundary is Lipschitz and that the value function is C2 in the interior of the no-action region. We then use a verification argument applied to a suitable C2 approximation of the value function to establish optimality of the conjectured control.

Article information

Ann. Appl. Probab., Volume 18, Number 6 (2008), 2367-2391.

First available in Project Euclid: 26 November 2008

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 93E20: Optimal stochastic control 60K25: Queueing theory [See also 68M20, 90B22] 60G40: Stopping times; optimal stopping problems; gambling theory [See also 62L15, 91A60] 49J30: Optimal solutions belonging to restricted classes (Lipschitz controls, bang-bang controls, etc.) 49L25: Viscosity solutions 35J60: Nonlinear elliptic equations

Singular control state constraints Brownian control problems optimal stopping free boundary obstacle problems viscosity solutions Hamilton–Jacobi–Bellman equations stochastic networks


Budhiraja, Amarjit; Ross, Kevin. Optimal stopping and free boundary characterizations for some Brownian control problems. Ann. Appl. Probab. 18 (2008), no. 6, 2367--2391. doi:10.1214/08-AAP525.

Export citation


  • [1] Atar, R. and Budhiraja, A. (2006). Singular control with state constraints on unbounded domain. Ann. Probab. 34 1864–1909.
  • [2] Bather, J. and Chernoff, H. (1967). Sequential decisions in the control of a spaceship. In Proc. Fifth Berkeley Sympos. Mathematical Statistics and Probability (Berkeley, Calif., 1965/66) III: Physical Sciences 181–207. Univ. California Press, Berkeley.
  • [3] Beneš, V. E., Shepp, L. A. and Witsenhausen, H. S. (1980/81). Some solvable stochastic control problems. Stochastics 4 39–83.
  • [4] Benth, F. E. and Reikvam, K. (2004). A connection between singular stochastic control and optimal stopping. Appl. Math. Optim. 49 27–41.
  • [5] Budhiraja, A. and Ghosh, A. P. (2005). A large deviations approach to asymptotically optimal control of crisscross network in heavy traffic. Ann. Appl. Probab. 15 1887–1935.
  • [6] Budhiraja, A. and Ross, K. (2006). Existence of optimal controls for singular control problems with state constraints. Ann. Appl. Probab. 16 2235–2255.
  • [7] Caffarelli, L. A. (1977). The regularity of free boundaries in higher dimensions. Acta Math. 139 155–184.
  • [8] Chen, M., Pandit, C. and Meyn, S. (2003). In search of sensitivity in network optimization. Queueing Syst. 44 313–363.
  • [9] Evans, L. C. (1979). A second-order elliptic equation with gradient constraint. Comm. Partial Differential Equations 4 555–572.
  • [10] Gilbarg, D. and Trudinger, N. S. (1983). Elliptic Partial Differential Equations of Second Order, 2nd ed. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 224. Springer, Berlin.
  • [11] Harrison, J. M., Sellke, T. M. and Taylor, A. J. (1983). Impulse control of Brownian motion. Math. Oper. Res. 8 454–466.
  • [12] Harrison, J. M. and Wein, L. M. (1989). Scheduling networks of queues: Heavy traffic analysis of a simple open network. Queueing Systems Theory Appl. 5 265–279.
  • [13] Ishii, H. and Koike, S. (1983). Boundary regularity and uniqueness for an elliptic equation with gradient constraint. Comm. Partial Differential Equations 8 317–346.
  • [14] Karatzas, I. (1983). A class of singular stochastic control problems. Adv. in Appl. Probab. 15 225–254.
  • [15] Karatzas, I. and Shreve, S. E. (1984). Connections between optimal stopping and singular stochastic control. I. Monotone follower problems. SIAM J. Control Optim. 22 856–877.
  • [16] Karatzas, I. and Shreve, S. E. (1985). Connections between optimal stopping and singular stochastic control. II. Reflected follower problems. SIAM J. Control Optim. 23 433–451.
  • [17] Kinderlehrer, D. and Nirenberg, L. (1977). Regularity in free boundary problems. Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4) 4 373–391.
  • [18] Kumar, S. and Muthuraman, K. (2004). A numerical method for solving singular stochastic control problems. Oper. Res. 52 563–582.
  • [19] Lions, P.-L. and Sznitman, A.-S. (1984). Stochastic differential equations with reflecting boundary conditions. Comm. Pure Appl. Math. 37 511–537.
  • [20] Martins, L. F., Shreve, S. E. and Soner, H. M. (1996). Heavy traffic convergence of a controlled, multiclass queueing system. SIAM J. Control Optim. 34 2133–2171.
  • [21] Meyn, S. P. (2003). Sequencing and routing in multiclass queueing networks. II. Workload relaxations. SIAM J. Control Optim. 42 178–217 (electronic).
  • [22] Protter, P. E. (2004). Stochastic Integration and Differential Equations, 2nd ed. Applications of Mathematics (New York) 21. Springer, Berlin.
  • [23] Rockafellar, R. T. (1970). Convex Analysis. Princeton Univ. Press, Princeton, NJ.
  • [24] Shiryayev, A. N. (1978). Optimal Stopping Rules. Springer, New York.
  • [25] Soner, H. M. and Shreve, S. E. (1989). Regularity of the value function for a two-dimensional singular stochastic control problem. SIAM J. Control Optim. 27 876–907.
  • [26] Shreve, S. E. and Soner, H. M. (1991). A free boundary problem related to singular stochastic control. In Applied Stochastic Analysis (London, 1989). Stochastics Monographs 5 265–301. Gordon and Breach, New York.