We give a concise introduction to risk sensitive control of Markov diffusion processes and related two-controller, zero-sum differential games. The method of dynamic programming for the risk sensitive control problem leads to a nonlinear partial differential equation of Hamilton-Jacobi-Bellman type. In the totally risk sensitive limit, this becomes the Isaacs equation for the differential game. There is another interpretation of the differential game using the Maslov idempotent probability calculus. We call this a max-plus stochastic control problem. These risk sensitive control/differential game methods are applied to problems of importance sampling for Markov diffusions.

This paper gives comparison principles for first-order PDEs of the Hamilton-Jacobi-Bellman type that arise in the problem of filtering under unknown disturbances with set-membership bounds on the uncertainty. The exact solutions of this problem, given in set-theoretic terms as “information sets,” are expressed as level sets to the solutions of some specific types of the HJB equation. But these solutions require complicated calculations. This paper presents an alternative approach that avoids exact solutions in favor of their upper and lower bounds, which in many cases may suffice for solving the required problems. For systems with linear structure ellipsoidal estimates are given, which ensure tight approximations of the convex information sets.

For a simple model of a stock, whose stock price is a geometric Brownian motion in which the drift rate changes back and forth between positive and negative values, optimal selling times are computed. An infinite optimal selling time meaning you should just hold the stock.

A discrete time filter is constructed where both the observation and signal process have non-linear dynamics with additive white Gaussian noise. Using the reference probably frame- work a convolution Zakai equation is obtained which updates the unnormalized conditional density. Our work obtains approximate solutions of this equation in terms of Gaussian sum when second order expansions are introduced for the non-linear terms.

We consider stochastic dynamic games in large population conditions where multiclass agents are weakly coupled via their individual dynamics and costs. We approach this large population game problem by the so-called Nash Certainty Equivalence (NCE) Principle which leads to a decentralized control synthesis. The McKean-Vlasov NCE method presented in this paper has a close connection with the statistical physics of large particle systems: both identify a consistency relationship between the individual agent (or particle) at the microscopic level and the mass of individuals (or particles) at the macroscopic level. The overall game is decomposed into (i) an optimal control problem whose Hamilton-Jacobi-Bellman (HJB) equation determines the optimal control for each individual and which involves a measure corresponding to the mass effect, and (ii) a family of McKean-Vlasov (M-V) equations which also depend upon this measure. We designate the NCE Principle as the property that the resulting scheme is consistent (or soluble), i.e. the prescribed control laws produce sample paths which produce the mass effect measure. By construction, the overall closed-loop behaviour is such that each agent’s behaviour is optimal with respect to all other agents in the game theoretic Nash sense.