Functions of Processes with Markovian States

Abstract

Let $\{Y_k\}$ be a stochastic process where either $k = 1,2, \cdots$, or $k = 0, \pm 1, \cdots$. Let $S$ and $T$ be measurable sets of sequences of states. Let $\epsilon$ be a state. Assume $P(Y_n = \epsilon) > 0$. Let $p_n(S\epsilon T) = P((\cdots, Y_{n-2}, Y_{n-1}) \varepsilon S, Y_n = \epsilon, (Y_{n+1}, Y_{n+2}, \cdots) \varepsilon T)$. We define the rank of $\epsilon$ at time $n$ to be maximal rank of matrices $(p_n(S_i\epsilon T_j); i, j = 1, \cdots, m)$ as $m$, the $S_i$ and the $T_j$ vary. In the stationary case, since the rank does not depend on $n$, we will refer to the rank of $\epsilon$. In this case, with finite state space, Gilbert [6] denoted the rank of $\epsilon$ by $n(\epsilon)$. A state which has rank 1 at time $n$ is a Markovian state at time $n$. A stochastic process all of whose states are Markovian at all times is a Markov process. Let $\{X_k\}$ be a second stochastic process indexed as $\{Y_k\}$. Gilbert proved (but stated in far less generality) that if $\nu_n(\epsilon)$ and $\mu_n(\delta)$ are the ranks at time $n$ of the states $\epsilon$ in $\{Y_k\}$ and $\delta$ in $\{X_k\}$, respectively, and if $Y_n = f(X_n)$, then \begin{equation*}\tag{1.1}\nu_n(\epsilon) \leqq \sum_{f(\delta) = \epsilon} \mu_n(\delta).\end{equation*} In the stationary finite state space case, Dharmadhikari [2] gave conditions under which if $\{Y_k\}$ has all states of finite rank it is possible to express $Y_k = f(X_k)$ where $\{X_k\}$ is stationary and Markovian. An example was given to show equality need not hold in (1.1). A similar proof can be used to show in the more general case that if $\epsilon$ is a state of finite rank in $\{Y_k\}$ and his conditions hold for $\epsilon$, then it is possible to write $Y_k = f(X_k)$ where $\epsilon = f(\epsilon_i)$ for a finite number of states $\epsilon_i$ of $\{X_k\}$ which are Markovian and $\delta = f(\delta)$ for $\delta \neq \epsilon_i$. See [4]. In Section 2 of the present paper we present an example showing that finite rank alone does not guarantee Dharmadhikari's result. In this case we have stationarity for $\{X_k\}$ and $\{Y_k\}$ and $\epsilon = f(\delta)$ for a countable set of states $\delta$ of $\{X_k\}$ which are Markovian. This is an example referred to by Dharmadhikari [2] and disproves a conjecture due to Gilbert. In Section 3 it is proved that if $\{Y_k\}$ has a state $\epsilon$ of finite rank at time $n$, then there exists a stochastic process $\{X_k\}$ such that $Y_k = f(X_k)$ where $\delta = f(\delta)$ if $\delta \neq \epsilon_i$ and $\epsilon = f(\epsilon_i)$ for a countable family $\{\epsilon_i\}$ of Markovian states of $\{ X_k\}$ at time $n$. However, we are unable to prove that the ranks of states at times other than $n$ are undisturbed. Trivially, no rank may be decreased. Alternative constructions are given to show Markovianess and rank 2 of states may be preserved in $\{X_k\}$. The latter is deferred to Section 4. Our construction is valid both for $n = 0, \pm 1, \cdots$ and for $n = 1,2, \cdots$ without any assumption of finiteness of the state space of $\{Y_k\}$. Trivially, if this state space is countable and $n = 1,2, \cdots$ it is possible to represent $Y_k = f(X_k)$ where $\{X_k\}$ is a countable state Markov process. In this case Carlyle [1] gave a particular construction of $\{X_k\}$ which, in the case that $\{X_k\}$ has a finite state space, yields the minimal state space. In Section 4 we consider a state $\epsilon$ of rank 2 at time $n$. It is proved that in this case the construction of the process $\{X_k\}$ can be carried out so that there are only two states, $\epsilon_1$ and $\epsilon_2$, for which $\epsilon = f(\epsilon_i)$. Furthermore, it is proved that ranks of all states $\delta \neq \epsilon$ are undisturbed. If, in addition, $\{Y_k\}$ is stationary, it is proved in Section 5 that a stationary $\{ X_k\}$ can be constructed with all these properties. The definition of rank can be readily extended to the case of densities. All results of this paper as well as Gilbert's and Dharmadhikari's results can be obtained in this case. These considerations are deferred to a later paper. The results of Section 4 have found applications to a stochastic process describing the temporal behavior of cloud cover [5]. The observable states of cloud cover are clear (under 5% of the sky covered by clouds), partly cloudy (between 5% and 50% covered), cloudy (between 50% and 95% covered) and overcast (over 95% covered). A nonstationary Markov chain does not fit the data. Certain matrices of observed frequencies are, approximately, of rank 2 for data from Boston. Assume the observable process is a function of a nonstationary Markov chain with eight states, each state of the observable process being the image of two Markovian states. This model fits the Boston data very well.