The Decomposition-Separation Theorem for Finite Nonhomogeneous Markov Chains and Related Problems
Isaac M. Sonin
Abstract
Let M be a finite set, P be a stochastic matrix and U={(Zn)} be the family of all finite Markov chains (MC) (Zn) defined by M, P, and all possible initial distributions. The behavior of a MC (Zn) is a classical result of probability theory derived in the 1930s by A. N. Kolmogorov and W. Doeblin. If a stochastic matrix P is replaced by a sequence of stochastic matrices (Pn) and transitions at moment n are defined by Pn, then U becomes a family of nonhomogeneous MCs. There are numerous results concerning the behavior of such MCs given some specific properties of the sequence (Pn). But what if there are no assumptions about sequence (Pn)? Is it possible to say something about the behavior of the family U? The surprising answer to this question is Yes. Such behavior is described by a theorem which we call a decomposition- separation (DS) theorem, and which was initiated by a small paper of A. N. Kolmogorov (1936) and formulated and proved in a few stages in a series of papers including D. Blackwell (1945), H. Cohn (1971, 1989), and I. Sonin (1987, 1991, 1996).
First Page:
Show
Hide
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.imsc/1233152931
Digital Object Identifier: doi:10.1214/074921708000000264
References
[1] Alberti, P. and Uhlmann, A. (1982). Stochasticity and Partial Order: Doubly stochastic maps and unitary mixing. Mathematics and Its Applications 9. D. Reidel Publishing Co., Dordrecht, Boston.
Mathematical Reviews (MathSciNet):
MR667518
[2] Blackwell, D. (1945). Finite nonhomogeneous Markov chains. Ann. Math. 46 594–599.
Mathematical Reviews (MathSciNet):
MR13858
Digital Object Identifier: doi:10.2307/1969199
JSTOR: links.jstor.org
[3] Cohn, H. (1970). On the tail σ-algebra of the finite Markov chains. Ann. Math. Statist. 41 2175–2176.
Mathematical Reviews (MathSciNet):
MR268970
Zentralblatt MATH:
0217.50302
Digital Object Identifier: doi:10.1214/aoms/1177696725
Project Euclid: euclid.aoms/1177696725
[4] Cohn, H. (1976). Finite nonhomogeneous Markov chains: Asymptotic behavior. Adv. Appl. Prob. 8 502–516.
Mathematical Reviews (MathSciNet):
MR418242
Zentralblatt MATH:
0371.60082
Digital Object Identifier: doi:10.2307/1426141
JSTOR: links.jstor.org
[5] Cohn, H. and Fielding, M. (1999). Simulated annealing: Searching for an optimal temperature schedule. SIAM J. Optim. 9 (3) 779–802.
Mathematical Reviews (MathSciNet):
MR1700696
Zentralblatt MATH:
0957.60072
Digital Object Identifier: doi:10.1137/S1052623497329683
[6] Cohn, H. (1989). Products of stochastic matrices and applications. Int. J. Math. Sci. 12 209–333.
Mathematical Reviews (MathSciNet):
MR994904
Zentralblatt MATH:
0673.15010
Digital Object Identifier: doi:10.1155/S0161171289000268
[7] Dietz, Z. and Sethuraman, S. (2005). Large deviations for a class of nonhomogeneous Markov chains. Ann. Appl. Probab. 15 (1A) 421–486.
Mathematical Reviews (MathSciNet):
MR2115048
Zentralblatt MATH:
1072.60017
Digital Object Identifier: doi:10.1214/105051604000000990
Project Euclid: euclid.aoap/1106922333
[8] Dobrushin, R. (1956). Central limit theorem for non-stationary Markov chains. I. Theory Probab. Appl. 1 (1) 65–80.
Mathematical Reviews (MathSciNet):
MR86436
[9] Ethier, S. and Kurtz, T. (1986). Markov Processes: Characterization and Convergence. Wiley, New York.
Mathematical Reviews (MathSciNet):
MR838085
[10] Griffeath, D. (1975). Uniform coupling of non-homogeneous Markov chains. J. Appl. Probability 12 (4) 753–762.
Mathematical Reviews (MathSciNet):
MR386018
Zentralblatt MATH:
0322.60061
Digital Object Identifier: doi:10.2307/3212726
JSTOR: links.jstor.org
[11] Hajnal, J. (1956). The ergodic properties of non-homogeneous finite Markov chains. Proc. Cambridge Philos. Soc. 52 67–77.
Mathematical Reviews (MathSciNet):
MR73874
Digital Object Identifier: doi:10.1017/S0305004100030991
[12] Hartfiel, D. (2002). Nonhomogeneous Matrix Products. World Scientific Publishing Co., Inc., River Edge, NJ.
Mathematical Reviews (MathSciNet):
MR1878339
Zentralblatt MATH:
1011.15006
[13] Hill, T. (1979). On the existence of good Markov strategies. Trans. Am. Math. Soc. 247 157–176.
Mathematical Reviews (MathSciNet):
MR517690
Zentralblatt MATH:
0399.60046
Digital Object Identifier: doi:10.1090/S0002-9947-1979-0517690-9
[14] Iosifescu, M. (1966). On the uniform ergodicity of a class of nonhomogeneous random systems with complete connections. Rev. Roumaine Math. Pures Appl. 11 763–772.
Mathematical Reviews (MathSciNet):
MR214125
[15] Isaacson, D. and Madsen, R. (1976). Markov Chains: Theory and Applications. Wiley, New York.
Mathematical Reviews (MathSciNet):
MR407991
[16] Kemeny, J. and Snell, J. (1976). Finite Markov chains. Reprinting of the 1960 original. Springer-Verlag, New York, Heidelberg.
Mathematical Reviews (MathSciNet):
MR410929
[17] Kingman, J. F. C. (1975). Geometrical aspects of the theory of nonhomogeneous Markov chains. Math. Proc. Cambridge Philos. Soc. 77 171–183.
Mathematical Reviews (MathSciNet):
MR388541
Zentralblatt MATH:
0307.60064
Digital Object Identifier: doi:10.1017/S0305004100049501
[18] Kolmogoroff, A. N. (1936). Zur Theorie der Markoffschen Ketten. Math. Ann. 112 155–160. Selected Works of A. N. Kolmogorov 2 (A. N. Shiryaev, ed.). Probability Theory and Math. Statistics. Kluwer Acad. Publ.
Mathematical Reviews (MathSciNet):
MR1513044
Digital Object Identifier: doi:10.1007/BF01565412
[19] Liu, W. and Yang, W. (1996). An extension of Shannon–McMillan theorem and some limit properties for nonhomogeneous Markov chains. Stoch. Process. Appl. 61 (1) 129–145.
[20] Maksimov, V. M. (1970). The convergence of nonhomogeneous doubly stochastic Markov chains. Theory Probab. Appl. 15 622–636.
Mathematical Reviews (MathSciNet):
MR298773
[21] Marshall, A. and Olkin, I. (1979). Inequalities: Theory of Majorization and its Applications. Academic Press, New York.
Mathematical Reviews (MathSciNet):
MR552278
[22] Mukherjea, A. (1983). Nonhomogeneous Markov chains: Tail idempotents, tail sigma-fields and basis. Math. Z. 183 (3) 293–309.
Mathematical Reviews (MathSciNet):
MR706390
Digital Object Identifier: doi:10.1007/BF01176473
[23] Niemiro, W. and Pokarowski, P. (1995). Tail events of some nonhomogeneous Markov chains. Ann. Appl. Probab. 5 (1) 261–293.
Mathematical Reviews (MathSciNet):
MR1325053
Zentralblatt MATH:
0826.60055
Digital Object Identifier: doi:10.1214/aoap/1177004840
Project Euclid: euclid.aoap/1177004840
[24] Ruch, E. and Mead, A. (1976). The principle of increasing mixing character and some of its consequences. Journ. Theor. Chem. A. 41 (2) 95–117.
[25] Seneta, E. (1973). On the historical development of the theory of finite inhomogeneous Markov chains. Proc. Cambridge Philos. Soc. 74 507–513.
Mathematical Reviews (MathSciNet):
MR331522
Digital Object Identifier: doi:10.1017/S0305004100077276
[26] Sonin, I. (1987). Theorem on separation of jets and some properties of random sequences. Stochastics 21 231–250.
Mathematical Reviews (MathSciNet):
MR900114
Zentralblatt MATH:
0626.60070
[27] Sonin, I. M. (1988). The separation of jets and some asymptotic properties of random sequences. Discrete Event Systems: Models and Applications. Lecture Notes in Contr. Inf. 103 275–282. Springer.
Mathematical Reviews (MathSciNet):
MR947973
Zentralblatt MATH:
0645.60072
Digital Object Identifier: doi:10.1007/BFb0042318
[28] Sonin, I. M. (1991a). On an extremal property of Markov chains and sufficiency of Markov strategies in Markov decision processes with the Dubins–Savage criterion. Ann. of Oper. Res. 29 417–426.
Mathematical Reviews (MathSciNet):
MR1105185
Zentralblatt MATH:
0717.90098
Digital Object Identifier: doi:10.1007/BF02283608
[29] Sonin, I. (1991b). An arbitrary nonhomogeneous Markov chain with bounded number of states may be decomposed into asymptotically noncommunicating components having the mixing property. Theory Probab. Appl. 36 74–85.
Mathematical Reviews (MathSciNet):
MR1109017
[30] Sonin, I. (1996). The asymptotic behaviour of a general finite nonhomogeneous Markov chain (the decomposition-separation theorem). In Statistics, Probability and Game Theory, papers in Honor of David Blackwell (T. S. Ferguson, L. S. Shapley and J. B. MacQueen, eds.) Lecture Notes-Monograph Series, 30 337–346. Inst. of Math. Stat.
Mathematical Reviews (MathSciNet):
MR1481788
Digital Object Identifier: doi:10.1214/lnms/1215453581
[31] Sonin, I. (1997). On some asymptotic properties of nonhomogeneous Markov chains and random sequences with countable number of values. In Stat. and Control of Stoch. Proc., The Liptser Festschrift, Proceedings of Steklov Math. Inst. Seminar (Y. Kabanov, B. Rozovskii, A. Shiryaev, eds.) 297–313. World Sci. Publ., River Edge, NJ.
Mathematical Reviews (MathSciNet):
MR1647229
Zentralblatt MATH:
0952.60070
[32] Sethuraman, S. and Varadhan, S. R. S. (2005). A martingale proof of Dobrushin’s theorem for non-homogeneous Markov chains. Electron. J. Probab. 10 (36) 1221–1235.
[33] Shiryaev, A. N. (1996). Probability. Springer-Verlag, New York.
Mathematical Reviews (MathSciNet):
MR1368405
[34] Thorisson, H. (2000). Coupling, Stationarity, and Regeneration. Probability and its Applications. Springer-Verlag, New York.
Mathematical Reviews (MathSciNet):
MR1741181
Zentralblatt MATH:
0949.60007
[35] Vershik, A. and Kachurovski, A. (1999). Rates of convergence in ergodic theorems for locally finite groups, and reversed martingales. Differ. Equat. and Contr. Proc. (1) 19–26.
Institute of Mathematical Statistics Collections
