Estimating the Infinitesimal Generator of a Continuous Time, Finite State Markov Process

Abstract

Let $\{Z(t), t > 0\}$ be a separable, continuous time Markov Process with stationary transition probabilities $P_{ij}(t), i, j = 1, 2, \cdots, M$. Under suitable regularity conditions, the matrix of transition probabilities, $P(t)$, can be expressed in the form $P(t) = \exp tQ$, where $Q$ is an $M \times M$ matrix and is called the "infinitesimal generator" for the process. In this paper, a density on the space of sample functions over $[0, t)$ is constructed. This density depends upon $Q$. If $Q$ is unknown, the maximum likelihood estimate $\hat{Q}(k, t) = \|\hat{q}_{ij}(k, t)\|$, based upon $k$ independent realizations of the process over $\lbrack 0, t)$ can be derived. If each state has positive probability of being occupied during $\lbrack 0, t)$ and if the number of independent observations, $k$, grows larger ($t$ held fixed), then $\hat{q}_{ij}$ is strongly consistent and the joint distribution of the set $\{k^{\frac{1}{2}}(\hat{q}_{ij} - q_{ij})\}_{i \neq j}$ (suitably normalized), is asymptotically normal with zero mean and covariance equal to the identity matrix. If $k$ is held fixed (at one, say) and if $t$ grows large, then $\hat{q}_{ij}$ is again strongly consistent and the joint distribution of the set $\{t^{\frac{1}{2}}(\hat{q}_{ij} - q_{ij})\}_{i \neq j}$ (suitably normalized), is asymptotically normal with zero mean and covariance equal to the identity matrix, provided that the process $\{Z(t), t > 0\}$ is positively regular. The asymptotic variances of the $\hat{q}_{ij}$ are computed in both cases.