Source: Ann. Statist. Volume 32, Number 6
(2004), 2742-2775.
Given an arbitrary long but finite sequence of observations from a finite set, we construct a simple process that approximates the sequence, in the sense that with high probability the empirical frequency, as well as the empirical one-step transitions along a realization from the approximating process, are close to that of the given sequence.
We generalize the result to the case where the one-step transitions are required to be in given polyhedra.
References
Aldous, D. J. and Fill, J. A. (2002). Reversible Markov chains and random walks on graphs. Available at www.stat.berkeley.edu/users/aldous.
Alon, N., Spencer, J. H. and Erdös, P. (2000). The Probabilistic Method, 2nd ed. Wiley, New York.
Baum, L. E. and Eagon, J. A. (1967). An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology. Bull. Amer. Math. Soc. 73 360--363.
Mathematical Reviews (MathSciNet):
MR210217
Baum, L. E. and Petrie, T. (1966). Statistical inference for probabilistic functions of finite state Markov chains. Ann. Math. Statist. 37 1554--1563.
Mathematical Reviews (MathSciNet):
MR202264
Feller, W. (1968). An Introduction to Probability Theory and Its Applications 1, 3rd ed. Wiley, New York.
Mathematical Reviews (MathSciNet):
MR228020
Glynn, P. W. and Ormoneit, D. (2002). Hoeffding's inequality for uniformly ergodic Markov chains. Statist. Probab. Lett. 56 143--146.
Jerrum, M. and Sinclair, A. (1989). Approximating the permanent. SIAM J. Comput. 18 1149--1178.
Krogh, A., Mian, I. S. and Haussler, D. (1994). A hidden Markov model that finds genes in E. coli DNA. Nucleic Acids Research 22 4768--4778.
Lovász, L. and Kannan, R. (1999). Faster mixing via average conductance. In Proc. 31st Annual ACM Symposium on Theory of Computing 282--287. ACM, New York.
Lovász, L. and Simonovits, M. (1990). The mixing rate of Markov chains, an isoperimetric inequality, and computing the volume. In Proc. 31st Annual Symposium on Foundations of Computer Science 1 346--354. IEEE, Piscataway, NJ.
Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77 257--286.
Solan, E. and Vieille, N. (2003). Perturbed Markov chains. J. Appl. Probab. 40 107--122.