A GENERAL STOCHASTIC MAXIMUM PRINCIPLE FOR SINGULAR CONTROL PROBLEMS 1

We consider the stochastic control problem in which the control domain need not be convex, the control variable has two components, the ﬂrst being absolutely continuous and the second singular.The coe–cients of the state equation are non linear and depend explicitly on the absolutely continuous component of the control. We establish a maximum principle, by using a spike variation on the absolutely continuous part of the control and a convex perturbation on the singular one. This result is a generalization of Peng’s maximum principle to singular control problems.


Introduction
We consider in this paper stochastic control problems of nonlinear systems, where the control domain need not be convex and the control variable has two components, the first being absolutely continuous and the second singular. The system under consideration is governed by a stochastic differential equation of the following type where b, σ and G are given deterministic functions, x 0 is the initial state and B = (B t ) t≥0 is a standard Brownian motion, defined on a filtered probability space Ω, F, (F t ) t≥0 , P , satisfying the usual conditions. The control variable is a suitable process (u, ξ) where u : and ξ is an increasing process, continuous on the left with limits on the right with ξ 0 = 0.
The criteria to be minimized over the class of admissible controls has the form A control process that solves this problem is called optimal. We suppose that an optimal control exists. Our main goal in this paper is to establish necessary conditions for optimality of the Pontriagin type for this kind of problems.
Singular control problems have been studied by many authors including Benȇs, Shepp, and Witsenhausen [4] , Chow, Menaldi, and Robin [6] , Karatzas, Shreve [11] , Davis, Norman [7] , Haussmann, Suo [8,9,10] . See [8] for a complete list of references on the subject. The approaches used in these papers, to solve the problem are mainly based on dynamic programming. It was shown in particular that the value function is solution of a variational inequality, and the optimal state is a reflected diffusion at the free boundary. Note that in [8] , the authors apply the compactification method to show existence of an optimal singular control.
The other major approach to solve control problems is to derive necessary conditions satisfied by some optimal control, known as the stochastic maximum principle. The first version of the stochastic maximum principle that covers singular control problems was obtained by Cadenillas and Haussmann [5], in which they consider linear dynamics, convex cost criterion and convex state constraints. The method used in [5] is based on the known principle of convex analysis, related to the minimization of convex, Gâteaux-differentiable functionals defined on a convex closed set. Necessary conditions for optimality for non linear stochastic differential equations with convex state constraints or uncontrolled diffusion matrix and measure valued controls were obtained by Bahlali & Chala [1] and Bahlali & al [2] .
In our situation, since the system is nonlinear and the control domain is not necessarily convex, the approach of convex analysis used in [5] is no longer valid. Moreover, since the diffusion coefficient depends explicitly on the control variable, the method of first order variation used in [2] cannot be applied. The approach that we use to establish our main result is based on a double perturbation of the optimal control u, ξ . The first perturbation is a spike variation, on the absolutely continuous part of the control and the second one is convex, on the singular component. This perturbation is defined as follows where v is a A 1 -valued, F t −measurable random variable and η is an increasing process with η 0 = 0. The variational inequality is derived from the following inequality From the definition of our perturbation, it is difficult to derive directly the variational inequality. To handle this problem, it is necessary to separate the above inequality into two parts. Let The variational inequality will be obtained from the fact that For the singular part of the control, we apply the method of Bensoussan [3] , to derive a first order adjoint process and a variational inequality which reduces to the computation of a Gâteaux derivative. For the absolutely continuous part, we use the approach developed by Peng [12] to derive the first and second order adjoint processes and the second variational inequality. Putting together the adjoint processes and the variational inequalities, we obtain the stochastic maximum principle. Our result may be regarded as a generalization of Peng's maximum principle, to singular control problems.
Let us briefly describe the contents of this paper. In the second section, we formulate the problem and give the various assumptions used throughout the paper. The third section is devoted to some preliminary results, which will be used in the sequel. In the fourth section, we derive explicitly the first and second order adjoint processes and the variational inequalities. In the last section, we state the stochastic maximum principle which is our main result.

Formulation of the problem
Let Ω, F, (F t ) t≥0 , P be a probability space equipped with a filtration satisfying the usual conditions, on which a d-dimensional Brownian motion B = (B t ) t≥0 is defined. We assume that (F t ) is the P -augmentation of the natural filtration of (B t ) t≥0 .
Let T be a strictly positive real number and consider the following sets A 1 is a non empty subset of R k and A 2 = ([0, ∞)) m . U 1 is the class of measurable, adapted processes u : [0, T ] × Ω −→ A 1 . U 2 is the class of measurable, adapted processes ξ : [0, T ] × Ω −→ A 2 such that ξ is nondecreasing, left-continuous with right limits and ξ 0 = 0.
We denote by U the set of all admissible controls.
For any (u, ξ) ∈ U, we consider the following stochastic equation The expected cost has the form where The control problem is to minimize the functional J (.) over U. If u, ξ ∈ U is an optimal solution, that is we may ask, how we can characterize it, in other words what conditions must u, ξ necessarily satisfy?
The following assumptions will be in force throughout this paper b, σ, g, h are twice continuously differentiable with respect to x.
(3) The derivatives b x , b xx , σ x , σ xx , g x , g xx , h x , h xx are continuous in (x, u) and uniformly bounded. b, σ are bounded by C (1 + |x| + |u|) . G and k are continuous and G is bounded.
Under the above hypothesis, for every (u, ξ) ∈ U, equation (1) has a unique strong solution given by We list some matrix notation that will be used throughout this paper. We denote by M n×n (R) the space of n × n real matrix and M d n×n (R) the linear space of vectors For any M, N ∈ M d n×n (R), L ∈ M n×n (R), x, y ∈ R n and z ∈ R d , we use the following notation We denote by L * the transpose of the matrix L and M * = (M * 1 , ..., M * d ) .

Preliminary Results
The purpose of the stochastic maximum principle is to find necessary conditions for optimality satisfied by an optimal control. Suppose that u, ξ ∈ U is an optimal control and x (t) denotes the optimal trajectory, that is, the solution of (1) corresponding to u, ξ . Let us introduce the following perturbation of the optimal control u, ξ : where 0 ≤ τ < T is fixed, θ > 0 is sufficiently small, v is a F t −measurable random variable and η is an increasing process with η 0 = 0.
Since u, ξ is optimal, we have 0 ≤ J u θ , ξ θ − J u, ξ . Let The variational inequality will be derived from the fact that be the trajectories associated respectively with u θ , ξ θ , u θ , ξ . For simplicity of notation, we denote We will proceed by separating the computation of the two limits in (7) , and obtain a variational equality from (5) and a variational inequality from (6). To achieve this goal, we need the following technical lemmas.
where z is the solution of the linear stochastic differential equation Proof. From (3) , (4) and by using Gronwall's lemma and Burkholder Davis Gundy inequality, we get Let where ρ θ (t) is given by b x , σ x being continuous and bounded, then using (10) , (11) , (12) and the dominated convergence theorem, we get We conclude by using Gronwall's lemma.

Lemma 3 Under assumption (3), the following estimate holds
where x 1 , x 2 are solutions of We put It is clear that By using the same proof as in [12], lemma 1 page 968, we show that which prove the lemma. (14) and (15) are called the first and the second-order variational equations. Equation (14) is the variational equation in the usual sense. Since the diffusion coefficient σ depends explicitly on the control variable u and the control domain U is not convex, then by using only equation (14) we can only obtain the following estimation

Remark 4 Equations
This does not allow us to derive the variational inequality. The idea introduced by S. Peng [12] is to use a second-order expansion to obtain an estimation of order o (θ 2 ).

Lemma 5 Under assumptions of lemma 2, we have
Since g x and h x are continuous and bounded, then from (4) , (8) , (11) and by letting θ going to zero we conclude.
Lemma 6 Under assumptions of lemma 3 we have Proof. From (6) we have By using the estimate (13), the result follows by mimicking the same proof as in [24] (lemma 2, page 970).

Variational inequalities and adjoint processes
In this section, we introduce the adjoint processes and we derive the variational inequalities from (16) and (17). The backward stochastic differential equations satisfied by the adjoint processes will be given in the next section.

The first-order expansion
The linear terms in (16) and (17) may be treated in the following way (see Bensoussan [6]). Let Φ 1 be the fundamental solution of the linear equation This equation is linear with bounded coefficients, then it admits a unique strong solution. This solution is invertible and its inverse Ψ 1 (t) is the unique solution of the following equation Moreover, Φ 1 and Ψ 1 satisfy We introduce the following processes We remark from (21) , (22) , (23) , (24) that Since g x and h x are bounded, then from (20), X 1 is square integrable. Hence t≥0 is a square integrable martingale with respect to the natural filtration of the Brownian motion (B t ). Then from Ito's representation theorem we have where Q 1 (s) is an adapted process such that E T 0 |Q 1 (s)| 2 ds < ∞.
By applying the Ito's formula to α 1 (t) and β 1 (t) and using (25) and (26) , we can rewrite (16) and (17) as where p 1 and q 1 are adapted processes given by and the Hamiltonian H is defined from where σ i and q i denote respectively the i th columns of matrices σ and q.
The process p 1 is called the first order adjoint process and from (23) , (24) , (29), it is given explicitly by where Φ 1 (t) and Ψ 1 (t) are respectively the solutions of (18) and (19) .

The second-order expansion
We now treat the quadratic terms of (28) by the same method. Let Z = x 1 x * 1 , by Ito's formula we obtain where A θ (t) and B θ (t) are given by We consider now the following symmetric matrix-valued linear equation This equation is linear with bounded coefficients, hence it admits a unique strong solution. Φ 2 (t) is invertible and its inverse Ψ 2 (t) is the solution of the following equation Moreover, Φ 2 and Ψ 2 satisfy We put α 2 (t) = Ψ 2 (t)Z(t) (35) We remark from (35), (36) and (37) that Since g xx and H xx are bounded, then from (34), X 2 is square integrable, hence E [X 2 / F t ] is a square integrable martingale with respect to the Brownian filtration. Then from Ito's representation theorem, we have Where Q 2 (s) is an adapted process such that E T 0 |Q 2 (s)| 2 ds < ∞.