Open Access
Translator Disclaimer
October, 1979 Persistently Optimal Plans for Nonstationary Dynamic Programming: The Topology of Weak Convergence Case
Robert P. Kertz, David C. Nachman
Ann. Probab. 7(5): 811-826 (October, 1979). DOI: 10.1214/aop/1176994940


In this paper, we study a nonstationary dynamic programming model $\{(S_n, \mathscr{S}_n), (A_n, \mathscr{A}_n), D_n, q_n, u: n \geqslant 1\}$ with standard Borel state spaces $(S_n, \mathscr{S}_n)$ and action spaces $(A_n, \mathscr{A}_n)$, upper semicontinuous admissible-action correspondences $D_n$, weakly continuous transition laws $q_n$, and Borel measurable total reward function $u: S_1 \times A_1 \times \cdots \rightarrow R_-$. We establish existence of a persistently optimal (degenerate) plan for this model under regularity and boundedness assumptions on conditional expectations of $u$, but require no special separable form of $u$ such as intertemporal additivity. The methods of proof utilize results on weak convergence of probability measures and selection theorems in the context of optimization of functions over correspondences. We give two characterizations of the persistent optimality of a feasible plan: the first that the plan be both thrifty and equalizing, and the second that the plan satisfy an optimality criterion that entails period-by-period optimality.


Download Citation

Robert P. Kertz. David C. Nachman. "Persistently Optimal Plans for Nonstationary Dynamic Programming: The Topology of Weak Convergence Case." Ann. Probab. 7 (5) 811 - 826, October, 1979.


Published: October, 1979
First available in Project Euclid: 19 April 2007

zbMATH: 0417.49039
MathSciNet: MR542131
Digital Object Identifier: 10.1214/aop/1176994940

Primary: 49C15
Secondary: 60B10 , 60K99 , 62C05 , 90C99 , 93C55

Keywords: gambling , general expected utility criterion , maximization and selection theorems , Nonstationary discrete-time dynamic programming , optimality criterion , optimality equations , persistently optimal plan , weak convergence of probability measures

Rights: Copyright © 1979 Institute of Mathematical Statistics


Vol.7 • No. 5 • October, 1979
Back to Top