Martingale problems for conditional distributions of Markov processes

Let $X$ be a Markov process with generator $A$ and let $Y(t)=\gamma (X(t))$. The conditional distribution $\pi_t$ of $X(t)$ given $\sigma (Y(s):s\leq t)$ is characterized as a solution of a filtered martingale problem. As a consequence, we obtain a generator/martingale problem version of a result of Rogers and Pitman on Markov functions. Applications include uniqueness of filtering equations, exchangeability of the state distribution of vector-valued processes, verification of quasireversibility, and uniqueness for martingale problems for measure-valued processes. New results on the uniqueness of forward equations, needed in the proof of uniqueness for the filtered martingale problem are also presented.


Introduction
Suppose Z is a Markov process with state space E and that γ is a function mapping E into another space E 0 .We are interested in characterizing the conditional distribution of Z given the information obtained by observing the process Y defined by Y (t) = γ(Z(t)).This problem is, of course, the fundamental problem of filtering theory, but it is also closely related to such issues as identification of conditions under which Y is a Markov process and verification of quasireversibility in queueing networks.We approach these questions at a general level, characterizing the process Z as the solution of a martingale problem.The fundamental result is the characterization of the desired conditional distribution as the solution of another martingale problem.
Throughout, E will be a complete, separable metric space, B(E) will denote the bounded, Borel measurable functions on E, and P(E) the Borel probability measures on E. Let A be a mapping A : D(A) ⊂ B(E) → B(E), and ν 0 ∈ P(E).(Note that we do not assume that A is linear; however, usually it is or can easily be extended to be linear.)In general, we will denote the domain of an operator A by D(A) and its range by R(A).
A progressively measurable, E-valued process Z is a solution of the martingale problem for (A, ν 0 ) if ν 0 = P Z(0) −1 and there exists a filtration {F t } such that f(Z(t)) − t 0 Af(Z(s))ds is an {F t }-martingale for each f ∈ D(A).A measurable P(E)-valued function ν on [0, ∞) is a solution of the forward equation for (A, ν 0 ) if for each f ∈ D(A).(For µ ∈ P(E), we set µf = E fdµ.)Note that if Z is a solution of the martingale problem for (A, ν 0 ), then the one-dimensional distributions give a solution of the forward equation for (A, ν 0 ).A critical aspect of our discussion is conditions under which the converse of this statement holds, that is, given a solution of the forward equation ν, when does there exist a solution of the martingale problem Z satisfying (1.1).Let E 0 be a second complete, separable metric space, and let γ : E → E 0 be Borel measurable.Let Z be a solution of the martingale problem for (A, ν 0 ) defined on a complete probability space (Ω, F , P ), and define Y (t) = γ(Z(t)).We are interested in the filtering problem of estimating Z from observations of Y .Because we don't want to place regularity assumptions on Z and γ, we formulate this problem as follows: Let F Y t be the completion of σ( s 0 h(Y (u))du : s ≤ t, h ∈ B(E 0 )), and define π by π t (Γ) = P {Z(t) ∈ Γ| F Y t }. π can be taken to be an { F Y t }-progressive (in fact, { F Y t }-optional), P(E)-valued process.(See the appendix.)Note that is an { F Y t }-martingale for each f ∈ D(A) and that for each t ≥ 0 and h ∈ B(E 0 ), Furthermore, if Y is cadlag and has no fixed points of discontinuity, that is, P {Y (t) = Y (t−)} = 1 for all t ≥ 0, then F Y t equals the completion of σ(Y (s) : s ≤ t) for all t ≥ 0 and (1.3) can be replaced by Our primary goal is to give conditions underwhich (1.2) and (1.3) along with the requirement that E[π 0 ] = ν 0 determine the joint distribution of (Y, π).To be precise, we will say that ( Ỹ , π) is a solution of the filtered martingale problem for (A, ν 0 , γ), if E[π 0 ] = ν 0 , for each h ∈ B(E) and t ≥ 0, 2 Uniqueness for the forward equation The primary hypothesis of our main results will be uniqueness for the forward equation for (A, ν 0 ).There are a variety of conditions that imply this uniqueness, and if D(A) is separating, uniqueness for the forward equation implies uniqueness for the martingale problem.(See Ethier and Kurtz (1986), Theorem 4.4.2.)We identify A with its graph A = {(f, Af) : f ∈ D(A)}.See Ethier and Kurtz (1986) for definitions and results on generators and semigroups.
Let A ⊂ B(E)×B(E) and let A S be the linear span of A. We will say that A is dissipative if and only if A S is dissipative, that is, for (f, g) ∈ A S and λ > 0, λf − g ≥ λ f .
Note that a solution of the martingale problem (forward equation) for A will be a solution of the martingale problem (forward equation) for A S .
We say that A ⊂ B(E) × B(E) is a pre-generator if A is dissipative and there are sequences of functions µ n : E → P(E) and for each x ∈ E. Note that we have not assumed that µ n and λ n are measurable.
denotes the bounded continuous functions on E) and for each x ∈ E, there exists a solution ν x of the forward equation for (A, δ x ) that is right-continuous (in the weak topology) at zero, then A is a pre-generator.In particular, if (f, g) ∈ A, then which implies λf − g ≥ λf(x) and hence dissipativity, and if we take (We do not need to verify that ν x t is a measurable function of x for either of these calculations.) If E is locally compact and D(A) ⊂ Ĉ(E) ( Ĉ(E), the continuous functions vanishing at infinity), then the existence of λ n and µ n satisfying (2.1) implies A is dissipative.In particular, A S will satisfy the positive maximum principle, that is, if (f, g) ∈ A S and f(x 0 ) = f , then g(x 0 ) ≤ 0 which implies , and A satisfies the positive maximum principle, then A is a pre-generator.If E is locally compact, A ⊂ Ĉ(E) × Ĉ(E), and A satisfies the positive maximum principle, then A can be extended to a pre-generator on E ∆ , the one-point compactification of E. See Ethier and Kurtz (1986), Theorem 4.5.4.
Suppose A ⊂ C(E) × C(E).If D(A) is convergence determining, then every solution of the forward equation is continuous.Of course, if for each x ∈ E there exists a cadlag solution of the martingale problem for A, then there exists a right continuous solution of the forward equation, and hence, A is a pre-generator.
Theorem 2.1 (Semigroup conditions.)Let A ⊂ B(E) × B(E) be linear and dissipative.Suppose that there exists A ⊂ A such that R(λ − A ) ⊃ D(A ) for some λ > 0 (where the closure of A is in the sup norm) and that D(A ) is separating.Let A denote the closure in the sup norm of A .Then T (t)f = lim n→∞ (I − 1 n A ) −[nt] f defines a semigroup of linear operators on L = D(A ).If ν is a solution of the forward equation for A (and hence for A ), then ν t f = ν 0 T (t)f, f ∈ D(A ) and uniqueness holds for the forward equation and for the martingale problem for (A, ν 0 ).Remark 2.2 Proposition 4.9.18 of Ethier and Kurtz (1986) gives closely related conditions based on the assumption that R(λ − A) is separating for each λ > 0, but without assuming that D(A) is separating.
Proof.Existence of the semigroup follows by the Hille-Yosida theorem.Uniqueness for the martingale problem follows by Theorem 4.4.1 of Ethier and Kurtz (1986), and uniqueness for the forward equation follows by a similar argument.In particular, integration by parts gives Since if the range condition holds for some λ > 0 it holds for all λ > 0 (see Ethier and Kurtz (1986), Lemma 1.2.3), (2.2) implies for h ∈ D(A ) and hence, iterating this identity, where S [nt] is the sum of [nt] independent unit exponential random variables.Letting n → ∞, the law of large numbers and the continuity of ν t h for h ∈ D(A ) implies

2
The bp-closure of the set Theorem 2.3 Let A ⊂ B(E) × B(E) be linear and dissipative.Suppose that for each ν 0 ∈ P(E), there exists a cadlag solution of the martingale problem for (A, ν 0 ) and that there exists a λ > 0 for which R(λ−A) is bp-dense in B(E).Then for each ν 0 ∈ P(E), uniqueness holds for the martingale problem for (A, ν 0 ) and for the forward equation for (A, ν 0 ).
Proof.Let X x denote a cadlag solution of the martingale problem for (A, δ x ).Then for h ∈ R(λ − A), that is for h = λf − g for some (f, g) ∈ A, (2.3) (cf. (4.3.20) in Ethier and Kurtz (1986)).Let Ā denote the bp-closure of A. Note that if h n = λf n − g n for (f n , g n ) ∈ A and bp − lim n→∞ h n = h, then bp − lim n→∞ f n = f, where f is given by (2.3).It follows that R(λ − Ā) = B(E).Since Ā is still dissipative, by (2.3), we must have R(λ − Ā) = B(E) for all λ > 0. Since a process is a solution of the martingale problem for A if and only if it is a solution of the martingale problem for Ā, we may as well assume as λ → ∞.Consequently, we see that D( Ā) is bp-dense in B(E) and hence is separating.Consequently, the theorem follows by Theorem 2.1.

2
As noted above, uniqueness for the forward equation typically implies uniqueness for the martingale problem.The converse of this assertion does not hold in general.For example, let E = [0, 1], and Af(x) = 1 2 f (x).Then for any ν 0 ∈ P[0, 1], the unique solution of the martingale problem for (A, ν 0 ) is reflecting Brownian motion on [0, 1].Note, however, that ν t (dx) = 3I [ 1 3 , 2 3 ] (x)dx is a stationary solution of the forward equation that does not correspond to a solution of the martingle problem.In particular, the only stationary distribution for the martingale problem is the uniform distribution on [0, 1].(See Ethier and Kurtz (1986), Problem 4.11.4.)We next consider conditions under which the converse does hold.We will need the following separability hypothesis.
Hypothesis 2.4 There exists a countable subset {g k } ⊂ D(A) ∩ C(E) such that the graph of A is contained in the bounded, pointwise closure of the linear span of {(g k , Ag k )}.
Remark 2.5 If L ⊂ C(E) is separable and A ⊂ L × L, Hypothesis 2.4 is satisfied with the bounded, pointwise closure replaced by the (sup-norm) closure.In particular, if E is locally compact and A ⊂ Ĉ(E) × Ĉ(E), then the hypothesis is satisfied. If and a ij and b i are bounded on compact sets, then the hypothesis is satisfied.(Note that {f, If Hypothesis 2.4 holds and we define A 0 = {(g k , Ag k ) : k = 1, 2, . ..}, then any solution of the martingale problem (forward equation) for A 0 will be a solution of the martingale problem (forward equation) for A. (See Ethier and Kurtz (1986) Proposition 4.3.1.)For some of the consequences of Hypothesis 2.4, see Ethier and Kurtz (1986), Theorem 4.6.
Theorem 2.6 (Conditions based on the martingale problem.)Let A ⊂ C(E) × C(E) be a pre-generator and satisfy the separability Hypothesis 2.4.Suppose that D(A) is closed under multiplication (that is, f 1 , f 2 ∈ D(A) implies f 1 f 2 ∈ D(A)) and separates points.Then a) Each solution of the forward equation for A corresponds to a solution of the martingale problem for A.
b) If π ∈ P(E) satisfies E Afdπ = 0 for each f ∈ D(A), (that is, ν t ≡ π is a stationary solution of the forward equation), then there is a stationary solution of the martingale problem for (A, π).
c) If uniqueness holds for the martingale problem for (A, ν 0 ), then uniqueness holds for the forward equation for (A, ν 0 ).
Proof.This result is essentially Theorem 4.1 of Bhatt and Karandikar (1993), or if E is locally compact and A ⊂ Ĉ(E) × Ĉ(E), Proposition 4.9.19 of Ethier and Kurtz (1986).The technical modifications of the earlier results are discussed in the appendix.

2
Theorem 2.6c can be extended to any situation in which one can show that each solution of the forward equation corresponds to a solution of the martingale problem in the sense that the solution of the forward equation gives the one-dimensional distributions of the solution of the martingale problem.Uniqueness for the martingale problem then implies uniqueness for the forward equation.As noted previously, uniqueness for the forward equation for every initial distribution implies uniqueness for the martingale problem (Ethier and Kurtz (1986), Theorem 4.4.2).
A result of Kurtz and Stockbridge (1997) weakens the continuity hypothesis on the range of A in Theorem 2.6.
Theorem 2.7 Let E and F be complete and separable, and let A 0 ⊂ C(E) × C(E × F ). Let η be a transition function η from E to F , and define Suppose that D(A 0 ) is closed under multiplication and separates points, that for each y ∈ F , A y f ≡ A 0 f(•, y) is a pre-generator, and that A η satisfies Hypothesis 2.4.Then a) Each solution of the forward equation for A η corresponds to a solution of the martingale problem for ), (that is, ν t ≡ π is a stationary solution of the forward equation), then there is a stationary solution of the martingale problem for (A η , π).
c) If uniqueness holds for the martingale problem for (A η , ν 0 ), then uniqueness holds for the forward equation for (A η , ν 0 ).
Remark 2.8 a) Note that we are not assuming that A η f is continuous.In particular, any diffusion operator with bounded coefficients can be represented in this form.(With a little more care, the boundedness assumption on the coefficients can be relaxed.)b) Under the above conditions existence for the forward equation implies existence for the martingale problem and uniqueness for the forward equation for every initial distribution implies uniqueness for the martingale problem.
Proof.The result extends Theorem 3.1 of Kurtz and Stockbridge (1997) to the setting of Bhatt and Karandikar (1993) eliminating the assumption of local compactness.See also Bhatt and Borkar (1996).Technical details are discussed in the appendix.

2
In the development that follows, we will need to supplement the process Z with additional components obtained by solving differential equations of the form and we can take the state space of (Z, U) to be E × [0, 1].If Z is a solution of the martingale problem for A, then (Z, U) is a solution of the martingale problem for Â defined as follows: let The following theorem will be needed.
Theorem 2.9 Let If uniqueness holds for the martingale problem for (A, ν 0 ), then uniqueness holds for the martingale problem for ( Â, µ 0 ).b) If A satisfies the conditions of Theorem 2.3, then the linear span of Â satisfies the conditions of Theorem 2.3 and existence and uniqueness hold for the forward equation for Â and for the martingale problem for Â.
c) If A satisfies the conditions of Theorem 2.6 and γ is continuous, then Â satisfies the conditions of Theorem 2.6.d) If A satisfies the conditions of Theorem 2.6 or if A = A η where and A 0 and η satisfy the conditions of Theorem 2.7, then (without assuming continuity of γ) there exists Ã0 and η satisfying the conditions of Theorem 2.7 such that Âf(x, u) = F Ã0 f(x, u, y)η(x, u, dy).In particular, if uniqueness holds for the martingale problem for (A, ν 0 ) and µ 0 ∈ P(E × [0, 1]) has E-marginal ν 0 , then uniqueness holds for the forward equation for ( Â, µ 0 ).
Proof.Let (Z, U) be a solution of the martingale problem for ( Â, µ 0 ).By the same argument used in the proof of Ethier and Kurtz (1986), Theorem 4.3.6,U has a cadlag modification, which we will continue to denote by U. (Note that the assertions of the theorem only depend on the finite dimensional distributions of (Z, U).) It follows that for each t ≥ 0 and partitions 0 = and hence U is continuous.A similar calculation shows that which in turn implies This identity and Fubini's theorem imply that is determined by the finite dimensional distributions of Z, and hence, the finite dimensional distributions of (Z, U) are determined by the finite dimensional distributions of Z and part a) follows.Now consider part b).As in the proof of Theorem 2.3, we may as well assume that R(λ − A) = B(E).This condition implies that A is the full generator for the semigroup on B(E) determined by where X x is a solution of the martingale problem for (A, δ x ).(See Ethier and Kurtz (1986), Section 1.5.)Then (f, g) ∈ A if and only if Existence of cadlag solutions of the martingale problem for Â follows from existence for A and (2.4).Let ÂS denote the linear span of Â.To complete the proof, we need to verify Let f denote the partial derivative of f with respect to u.Let Âr be the collection of (f, g) approximate f by the Bernstein polynomial and set ), and set We need to show that for each u ∈ [0, 1], By (2.5) we must verify that (2.6) By the Markov property and differentiating by r verifies (2.6).
Finally, the collection of h satisfying the above properties is bp-dense in To verify part c), let and η(x, u, dy) = δ γ(x) (dy) .
Then Âf(x, u) = A η f(x, u) and the conditions of Theorem 2.7 are satisfied.
Then Âf(x, u) = Ãη f(x, u) and the conditions of Theorem 2.7 are satisfied.

2
3 Uniqueness for the filtered martingale problem . By the properties of Laplace transforms, it follows that the completion of σ( ).Let D( Â) be the collection of functions on Ê given by

and define Â by
The proof of the following lemma is essentially the same as the proof of Theorem 2.9.
Lemma 3.1 Let A and Â be as above.a) If uniqueness holds for the martingale problem for A, then uniqueness holds for the martingale problem for Â.
b) If A satisfies the conditions of Theorem 2.3, 2.6 or 2.7, then Â will satisfy the conditions of one of these theorems.In particular, if A satisfies the conditions of Theorem 2.3, 2.6, or 2.7, then each solution of the forward equation for Â corresponds to a solution of the martingale problem for Â, and hence, uniqueness for the martingale problem implies uniqueness for the forward equation.
Theorem 3.2 Let A ⊂ B(E) × B(E), ν 0 ∈ P(E), and γ : E → E 0 be Borel measurable.Suppose that each solution of the forward equation for ( Â, ν 0 × δ 0 ) corresponds to a solution of the martingale problem.(This condition will hold if A satisfies the conditions of Theorem 2.3, 2.6 or 2.7.)Let (π, Ỹ ) be a solution of the filtered martingale problem for (A, ν 0 , γ).
The following hold: a) There exists a solution Z of the martingale problem for (A, ν 0 ), such that Ỹ has the same distribution on b) For each t ≥ 0, there exists a Borel measurable mapping Note that for a.e.t, where ν0 = ν 0 × δ 0 and the last equality follows from the definition of ν and (3.3).Consequently, ν is a solution of the forward equation for ( Â, ν 0 × δ 0 ), and by assumption, there exists a solution (Z, U) of the martingale problem for ( Â, ν 0 × δ 0 ), such that (3.4) It follows that for each t, U(t) and Ũ(t) have the same distribution.Since F Y t equals the completion of σ(U(t)) and F Ỹ t equals the completion of σ( Ũ(t)), there exist mappings G t , Gt : [0, 1] ∞ → P(E) such that π t = G t (U(t)) a.s. and πt = Gt ( Ũ(t)) a.s.By (3.4) , where the last equality follows from the fact that U(t) and Ũ (t) have the same distribution.Applying (3.5) with g = G t (•)f and with g = Gt (•)f, we have and it follows that Consequently, πt f = G t ( Ũ(t))f a.s., and hence (π t , U(t)) has the same distribution as (π t , Ũ (t)).
Finally, if uniqueness holds for the martingale problem for (A, ν 0 ), then the distribution of (Z, Y, π) is uniquely determined and uniqueness holds for the filtered martingale problem for (A, ν 0 , γ).

2
We say that the filtered martingale problem for (A, γ) is well-posed if for each ν 0 ∈ P(E) there exists a unique (in the sense of Theorem 3.2(d)) solution of the filtered martingale problem for (A, ν 0 , γ).
Corollary 3.3 Let A satisfy the conditions of Theorem 2.3, and let γ : E → E 0 be Borel measurable.Then the filtered martingale problem for A is well-posed.
Proof.For each ν 0 ∈ P(E), existence holds for the martingale problem for (A, ν 0 ) by assumption, so existence holds for the filtered martingale problem for (A, ν 0 , γ).Theorem 2.3 and Theorem 2.9 imply A satisfies the conditions of Theorem 3.2 which establishes the corollary.

2
Corollary 3.4 Let A ⊂ B(E) × B(E), let γ : E → E 0 be Borel measurable.Suppose A satisfies the conditions of Theorem 2.6 or A = A η where and A 0 and η satisfy the conditions of Theorem 2.7.For each solution (π, Ỹ ) of the filtered martingale problem for (A, γ), there exists a solution Z of the martingale problem for A such that if Y = γ • Z and π t is the conditional distribution of Z(t) given F Y t , then for each 0 ≤ t 1 < • • • < t m , (π t 1 , . . ., πtm , Ỹ ) and (π t 1 , . . ., π tm , Y ) have the same distribution on P(E) m × M E 0 [0, ∞).If uniqueness holds for the martingale problem for (A, ν 0 ), then uniqueness holds for the filtered martingale problem for (A, ν 0 , γ).
Proof.Lemma 3.1 implies the conditions of Theorem 3.2 which establishes the corollary.2 The following corollaries address the question of when a function of a Markov process is Markov.For earlier results on this question, see Cameron (1973) and Rosenblatt (1966), Rogers and Pitman (1981), Kelly (1982), and Glover (1991).The results given here have application in proving uniqueness for martingale problems, in particular, for measure-valued processes.
Corollary 3.5 Let A ⊂ B(E) × B(E), and let γ : E → E 0 be Borel measurable.Let α be a transition function from Let µ 0 ∈ P(E 0 ), and define ν 0 = α(y, •)µ 0 (dy).Suppose that each solution of the forward equation for ( Â, ν 0 × δ 0 ) corresponds to a solution of the martingale problem.If Ỹ is a solution of the martingale problem for (C, µ 0 ), then there exists a solution Z of the martingale problem for (A, ν 0 ) such that Ỹ has the same distribution on and assume In particular, (3.6) is just (1) of Rogers and Pitman (1981).
The assumptions of part (b) imply Z is a Markov process (see Appendix A.4).By (3.6), P {Z(t) ∈ Γ|F Y t } = P {Z(t) ∈ Γ|Y (t)}, and since Z is Markov, which is the Markov property for Y 2 In the next corollary we consider martingale problems with "side conditions", that is, in addition to requiring Af(Z(s))ds to be a martingale for all f ∈ D(A) we require for a specified collection H ⊂ B(E).We will refer to this problem as the restricted martingale problem for (A, H, ν 0 ) (cf.Dawson (1993), Section 5).Of course, ν 0 must satisfy hdν 0 = 0, h ∈ H.We will denote the collection of probability measures satisfying this condition P H (E). The restricted forward equation has the obvious definition.The motivation for introducing this notion is a family of problems in infinite product spaces in which we want the coordinate random variables to be exchangeable.(See Dawson (1993), Donnelly andKurtz (1996, 1997).) Note that if A satisfies the conditions of Theorem 2.6 or 2.7, then Â satisfies the conditions of Theorem 2.7 and, consequently, if there exists a solution {µ t } of the restricted forward equation for ( Â, H), then there exists a solution (Z, U) of the martingale problem for Â that satisfies E[h(Z(t), U(t))] = µ t h for all h ∈ B(E × [0, 1] ∞ ).It follows that (Z, U) is a solution of the restricted martingale problem for ( Â, H), and if uniqueness holds for the restricted martingale problem for ( Â, H, µ 0 ), then uniqueness holds for the restricted forward equation.
Corollary 3.7 Let H ⊂ B(E), and let γ : E → E 0 be Borel measurable.Let α be a transition function from Let µ 0 ∈ P(E 0 ), and define ν 0 = α(y, •)µ 0 (dy).(Note that ν 0 ∈ P H (E).) Suppose that each solution of the restricted forward equation for ( Â, ν 0 × δ 0 ) corresponds to a solution of the restricted martingale problem.If Ỹ is a solution of the martingale problem for (C, µ 0 ), then there exists a solution Z of the restricted martingale problem for (A, H, ν 0 ) such that Ỹ has the same distribution on If, in addition, uniqueness holds for the restricted martingale problem for (A, H, ν 0 ), then uniqueness holds for the b) If uniqueness holds for the restricted martingale problem for (A, H, ν 0 ) and there exists a solution with sample paths in D E [0, ∞) or if uniqueness holds for the restricted martingale problem for (A, H, µ) for each µ ∈ P H (E), then Y is a Markov process.

Proof.
Let πt = α( Ỹ (t), •), and let Ũ be defined as in the proof of Theorem 3.2.Then (π, Ỹ , Ũ) is a solution of the filtered martingale problem for ( Â, ν 0 × δ 0 , γ) and νt = E[π t × δ Ũ(t) ] defines a solution of the restricted forward equation for ( Â, H, ν 0 × δ 0 ).By the hypotheses on A and Â, there exists a solution (Z, U) of the martingale problem for ( Â, ν 0 × δ 0 ) satisfying E[h(Z(t), U(t))] = hdν t (so it is also a solution of the restricted martingale problem), and as in the proof of Corollary 3.5, Ỹ has the same distribution on The original motivation for studying the filtered martingale problem comes from classical filtering theory, and the uniqueness theorem has been used to prove uniqueness for a variety of filtering equations.(See Kurtz and Ocone (1988), Kliemann, Koch and Marchetti (1990), Bhatt, Kallianpur, andKarandikar (1995), andFan (1996).)Theorem 3.2 and Corollary 3.4 can be used to improve slightly on these results.For example, the continuity assumption on h in Theorems 4. 1, 4.2, 4.5, and 4.6 of Kurtz and Ocone (1988) can be dropped.
Corollaries 3.5 and 3.7, however, provide the motivation for the present work, and the following examples illustrate their application.

Exchangeability.
Let A n be the generator of a Markov process with state space where Σ n is the collection of permutations of (1, . . ., n), and Theorem 4.1 Suppose A n satisfies the conditions of Theorem 2.6 or Theorem 2.7.For µ 0 ∈ P(P n (E)), define If there exists a solution of the martingale problem for (C n , µ 0 ), then there exists a solution X of the martingale problem for (A n , ν 0 ).If the solution of the martingale problem for (A n , ν 0 ) is unique, then the solution for (C n , µ 0 ) is unique and X satisfies In particular, (4.2) implies that for each t ≥ 0, (X 1 (t), . . ., X n (t)) is exchangeable.
Proof.The result is an immediate application of Corollary 3.5.

2
We illustrate Theorem 4.1 with the following simple application.Let B be the generator of a Markov process with state space E, let D( and where η ij (x) is the element of E n obtained by replacing x j by x i in x = (x 1 , . . ., x n ).A n 0 models a system of n particles that move independently in E according to the Markov process with generator B, live independent, exponentially distributed lifetimes, and at death are instantaneously replaced by a copy of one of the remaining n − 1 particles, selected at random.By symmetry, if (X 1 , . . ., X n ) is a solution of the martingale problem for A n 0 , then (X σ 1 , . . ., X σn ) is a solution, and if we define C n as in (4.1), it is easy to check that γ n (X) is a solution of the martingale problem for C n .
The generator is not symmetric; however, If the martingale problem for B is well-posed, then the martingale problems for A n 0 and A n will be well-posed.Theorem 4.1 implies that if X is a solution of the martingale problem for A n and X(0) = ( X1 (0), . . ., Xn (0)) is exchangeable, then for each t ≥ 0 X(t) is exchangeable and In addition, if X is a solution of the martingale problem for A n 0 , and X(0) has the same exchangeable distribution as X(0), then γ n (X) and γ n ( X) have the same distribution.See Donnelly andKurtz (1996, 1997) for further discussion and motivation for models of this type.

Uniqueness for measure-valued processes.
As above, let B be the generator of a Markov process with state space E. Let D(A) ⊂ B(E ∞ ) be given by D

}, and define
Note that since f ∈ D(A) depends on only finitely many components of x ∈ E ∞ , the sums in (4.5) are, in fact, finite.In addition, if X = (X 1 , X 2 , . ..) is a solution of the martingale problem for A, then X n = (X 1 , . . ., X n ) is a solution of the martingale problem for A n given by (4.3).Consequently, by the above results on exchangeability, if X(0) = (X 1 (0), X 2 (0), . ..) is exchangeable, then for each t ≥ 0, X(t) = (X 1 (t), X 2 (t), . ..) is exchangeable.(The (finite) exchangeability of (X 1 (t), . . ., X n (t)) for each n implies the (infinite) exchangeability of X(t).)Let If X(0) is exchangeable (so X(t) is exchangeable of all t ≥ 0), then by deFinetti's theorem where convergence is in the weak topology on P(E).Recall that Z n is a solution of the martingale problem for C n given by (4.4), and it follows that Z is a solution of the martingale problem for C ⊂ B(P(E)) × B(P(E)) given by where f, µ ∞ denotes integration of f by the product measure µ ∞ .Since the martingale problem for A is well-posed (assuming the martingale problem for B is well-posed) and exchangeability of X(0) implies exchangeability of X(t), it follows that the restricted martingale problem for (A, H) is well-posed, where H ⊂ B(E ∞ ) is the collection of all functions h of the form h(x 1 , x 2 , . ..) = f(x 1 , . . ., x m ) − f(x σ 1 , . . ., x σm ) for some f ∈ ∪ ∞ m=1 B(E m ) and some permutation (σ 1 , . . ., σ m ).Note that the restriction is just the requirement of the exchangeability of X(t).Fix η 0 ∈ P(E), and define γ : if the limit exists and γ(x) = η 0 otherwise.Define π t = Z ∞ (t).It follows that (Z, π) is a solution of the filtered martingale problem for (A, γ), and by Corollary 3.7, the solution of the martingale problem for C is well-posed.In particular, we have . Note that C is the generator for the Fleming-Viot process with mutation operator B. (See, for example, Ethier and Kurtz (1993).)For further applications of Corollary 3.7 to proofs of uniqueness of martingale problems for measure-valued processes, see Donnelly and Kurtz (1997).

Burke's theorem and quasireversibility.
One of the motivating examples for Corollary 3.5 is the proof of Burke's output theorem given by Kliemann, Koch, and Marchetti (1990).For the simplest example, let Q be an M/M/1 queue length process and D the corresponding departure process.Then the generator for Assuming λ < µ, the stationary distribution for Q is given by )), and we see that C in Corollary 3.5 is just the generator of the Poisson process with parameter λ.
Consequently, if D is a Poisson process with parameter λ, then (π 0 , D) is a solution of the filtered martingale problem for (A, π 0 , γ), and we have Burke's theorem.
Theorem 4.2 Let (Q, D) be the solution of the martingale problem for (A, π 0 × δ 0 ).Then D is a Poisson process with intensity λ, Q is stationary, and This theorem has been generalized in a variety of ways.For example, Serfozo (1989) gives conditions under which a variety of counting process functionals of a Markov chain are Poisson.Serfozo's arguments are based on time-reversal and Watanabe's characterization of a Poisson process.To see how these results can be obtained using Corollary 3.5, let be the generator of a pure-jump Markov process on E with stationary distribution π 0 .Let Φ ⊂ E × E − {(x, x) : x ∈ E} and ϕ : Φ → E .Let J(E ) be the collection of integer-valued measures on E .For a solution X of the martingale problem for A 1 , define a J(E )-valued process by that is, N(Γ, t) counts the number of times (X(s−), X(s)) ∈ Φ and ϕ(X(s−), X(s)) ∈ Γ.Then (X, N) is a solution of the martingale problem for As above, let γ(x, z) = z and α(z, Note that the second term on the right is zero since π 0 is a stationary distribution for A 1 .Suppose that there exists a measure β on E such that for each h ∈ B(E × E ), E×E λ(x)I Φ (x, y)h(y, ϕ(x, y))µ(x, dy)π 0 (dx) = E×E h(y, u)π 0 (dy)β(du).(4.6) Consequently, if X is a solution of the martingale problem for (A 1 , π 0 ), then (N, π 0 ) is a solution of the filtered martingale problem for (A, π 0 × δ 0 , γ) which implies N is a solution of the martingale problem for A 2 given by  (1989).Serfozo also shows that this condition is necessary for N to be Poisson and the independence property to hold.
4.4 Reflecting Brownian motion.Harrison andWilliams (1990, 1992) have considered analogues of the Poisson output theorems for Brownian network models.Here we give a version of their result for a single service station.
Let W = (W 1 , . . ., W m ) T be standard Brownian motion, and let X be the one-dimensional reflecting Brownian motion satisfying the Skorohod equation Consequently, if we take D(A) = {f ∈ C 2 c (R d+1 ) : Bf = 0}, then (X, Y ) is a solution for the martingale problem for A.
For β = 2b/ a 2 i , let π 0 (dx) = βe −βx dx.Then π 0 is the stationary distribution for X.As above, define α(y, •) = π 0 × δ y .Then , and the a i satisfy then the second term on the right of (4.7) is zero by the boundary condition Bf = 0. Let Ỹ be a solution of the martingale problem for with Ỹ (0) = 0. Then ( Ỹ , π 0 ) is a solution of the filtered martingale problem for (A, π 0 ×δ 0 , γ), γ(x, y) = y.It follows that if (X, Y ) is a solution of the martingale problem for (A, π 0 × δ 0 ), then Y is Brownian motion with generator A 0 and X(t) is independent of σ(Y (s) : s ≤ t).

A Appendix
A.1 Proof of Theorem 2.6 Let A satisfy the conditions of Theorem 2.6.Since A is a pre-generator, we can assume (1, 0) ∈ A. The domain of the linear span of A is then an algebra that separates points and vanishes nowhere.In addition, we can assume that {g k } is closed under multiplication.
Let I be the collection of finite subsets of positive integers, and for I ∈ I, let k(I) satisfy g k(I) = i∈I g i .For each k, there exists Note that Ê is compact.Following Bhatt and Karandikar (1993), define G : E → Ê by Then G has a measurable inverse defined on the (measurable) set G(E).We will need the following lemma.
Lemma A.1 Let ν ∈ P(E).Then there exists a unique measure µ ∈ P( Ê) satisfying Proof.Existence is immediate.Take µ = νG −1 .Since Ê is compact, { i∈I z i : I ∈ I} is separating.Consequently, uniqueness follows from the fact that 2 Suppose that µ ∈ P(E) satisfies Then there exists a stationary process that is a solution of the martingale problem for (A, µ).This assertion is essentially Theorem 3.1 of Bhatt and Karandikar (1993) with the assumption of existence of solutions of the D E [0, ∞) martingale problem replaced by the assumption that A is a pre-generator (which is, as we noted previously, implied by the existence of cadlag solutions).The proof of the earlier result only needs to be modified using the fact that a pre-generator is dissipative (needed in the definition of A n ) and the following generalization of Lemma 4.9.16 of Ethier and Kurtz (1986) (needed to show that Λ is a positive functional).Proof.Since A is a pre-generator, there exists λ n and µ n such that To complete the proof of Theorem 2.6, let ν be a solution of the forward equation, that is ν t f = ν 0 f + We assume, without loss of generality, that (S, Z) is defined for all t ∈ (−∞, ∞) on a probability space (Ω, F , P ), and define G t = σ(S(s), Z(s) : s ≤ t).Let τ −1 = sup{t < 0 : S(t) = 0}, τ 1 = inf{t > 0 : S(t) = 0}, and τ 2 = {t > τ 1 : S(t) = 0}.Taking ϕ n (s) = e −ns and applying the optional sampling theorem, it is easy to check that By Lemma A.1, it follows that P {Z(τ 1 ) ∈ G(E)} = 1 and Y (τ 1 ) has distribution ν 0 .
Following the argument in the proof of Theorem 4.1 in Kurtz and Stockbridge (1997),  can be obtained by including factors of the form g( t 0 h( Xr (s))ds) for t ≤ r and g( t r h( Xr (s))ds) for t ≥ r in the product.
Since Xr is a solution of the martingale problem for (A, ν 0 ), Xr and X have the same finite dimensional distributions.Note that the finite dimensional distributions of X determine the joint distribution of any collection of random variables of the form (X(t 1 ), . . ., X(t m ), t 1 0 h 1 (X(s))ds, . . ., tm 0 h m (X(s))ds), h 1 , . . ., h m ∈ B(E).(Compute joint moments of g i (X(t i )) and the integral terms.)Consequently, if H = H(X(t 1 ), . . ., X(t m ), .6) a) If, in addition, uniqueness holds for the martingale problem for (A, ν 0 ), then uniqueness holds for the M E 0 [0, ∞)-martingale problem for (C, µ 0 ).If Ỹ has sample paths in D E 0 [0, ∞), then uniqueness holds for the D E 0 [0, ∞)-martingale problem for (C, µ 0 ).b) If uniqueness holds for the martingale problem for (A, ν 0 ) then Y is a Markov process.Remark 3.6 Part b) is essentially a generator/martingale problem version of Theorem 2 of Rogers and Pitman (1981).Let P (t, z, •) be a transitition function corresponding to A. They define Q(t, y, Γ) = E P (t, z, γ −1 (Γ))α(y, dz), Γ ∈ B(E 0 ) of Part b) is the same as in Corollary 3.5.2 4 Applications.