Persistently Optimal Plans for Nonstationary Dynamic Programming: The Topology of Weak Convergence Case

Abstract

In this paper, we study a nonstationary dynamic programming model $\{(S_n, \mathscr{S}_n), (A_n, \mathscr{A}_n), D_n, q_n, u: n \geqslant 1\}$ with standard Borel state spaces $(S_n, \mathscr{S}_n)$ and action spaces $(A_n, \mathscr{A}_n)$, upper semicontinuous admissible-action correspondences $D_n$, weakly continuous transition laws $q_n$, and Borel measurable total reward function $u: S_1 \times A_1 \times \cdots \rightarrow R_-$. We establish existence of a persistently optimal (degenerate) plan for this model under regularity and boundedness assumptions on conditional expectations of $u$, but require no special separable form of $u$ such as intertemporal additivity. The methods of proof utilize results on weak convergence of probability measures and selection theorems in the context of optimization of functions over correspondences. We give two characterizations of the persistent optimality of a feasible plan: the first that the plan be both thrifty and equalizing, and the second that the plan satisfy an optimality criterion that entails period-by-period optimality.