Large Deviations Analysis of Some Recursive Algorithms with State Dependent Noise

Abstract

Consider the recursive, stochastic algorithm $X^\varepsilon_{n+1} = X^\varepsilon_n + \varepsilon b(X^\varepsilon_n, \xi_n)$, where $\{\xi_n\}$ is a random process and $X^\varepsilon_n$ lives in $\mathbb{R}^d$. Algorithms of this type arise frequently in applications in control and communications, as well as elsewhere. In the study of the important long term behavior of such recursive algorithms the "large deviations" behavior of the system, which describes the asymptotics of the order 1 deviations of the system from its "mean" trajectory as $\varepsilon$ tends to 0, plays a central role. Typical systems arising in communication theory and control often use complicated forcing terms involving correlated and state dependent noises and forcing terms with discontinuities. This paper presents a general approach for proving large deviations type theorems for such systems. The problem of proving such a theorem is considered first for the general case of a stochastic process with Lipschitz continuous sample paths. The assumptions are stated in terms of the conditional distribution of time increments of the process. After giving the proof in this general framework, we give several examples (in both continuous and discrete time) of driving terms that satisfy the hypotheses. The results are subsequently extended to a "projected" version of the discrete time model. The paper concludes with an application of the results to an automatic routing mechanism.