Nonparametric Estimation of Markov Transition Functions

Abstract

Let $\{X_n\}$ be a Markov chain having a stationary transition function and assume that the state set is an arbitrary set in a Euclidean space. The state transition law of the chain is given by a function $F(y|x) = P\lbrack X_{n+1} \leqslant y|X_n = x\rbrack$, which is assumed defined and continuous for all $x$. In this paper we give a statistical procedure for determining a function $F_n(y\mid x)$ on the basis of the sample $\{X_j\}^n_{j=1}, n = 1, 2,\cdots,$ and prove that if the chain is irreducible, aperiodic, and possesses a limiting distribution $\pi$, then with probability 1, $\sup_y|F_n(y|x) - F(y|x)| \rightarrow_n0$ for every $x$ such that any open sphere containing $x$ has positive $\pi$ probability. This result improves upon a study by Roussas which gives only weak convergence. We demonstrate that a certain clustering algorithm is useful for obtaining efficient versions of our estimates. The potential value of our methods is illustrated by computer studies using simulated data.