The stability of conditional Markov processes and Markov chains in random environments
Ramon van Handel
Source: Ann. Probab. Volume 37, Number 5 (2009), 1876-1925.
Abstract
We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this conditional signal is weakly ergodic when the signal is ergodic and the observations are nondegenerate. This permits a delicate exchange of the intersection and supremum of σ-fields, which is key for the stability of the nonlinear filter and partially resolves a long-standing gap in the proof of a result of Kunita [J. Multivariate Anal. 1 (1971) 365–393]. A similar result is obtained also in the continuous time setting. The proofs are based on an ergodic theorem for Markov chains in random environments in a general state space.
Primary Subjects: 93E11
Secondary Subjects: 60J05, 62M20, 93E15
Keywords: Nonlinear filtering; asymptotic stability; hidden Markov models; weak ergodicity; tail σ-field; exchange of intersection and supremum; Markov chain in random environment
Full-text: Access denied (no subscription detected)
We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber.
If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.aop/1253539859
Digital Object Identifier: doi:10.1214/08-AOP448
Zentralblatt MATH identifier:
05625055
References
[1] Baxendale, P., Chigansky, P. and Liptser, R. (2004). Asymptotic stability of the Wonham filter: Ergodic and nonergodic signals. SIAM J. Control Optim. 43 643–669.
Mathematical Reviews (MathSciNet):
MR2086177
Zentralblatt MATH:
1101.93074
Digital Object Identifier: doi:10.1137/S0363012902416924
[2] Bhatt, A. G., Budhiraja, A. and Karandikar, R. L. (2000). Markov property and ergodicity of the nonlinear filter. SIAM J. Control Optim. 39 928–949.
Mathematical Reviews (MathSciNet):
MR1786337
Zentralblatt MATH:
0970.60047
Digital Object Identifier: doi:10.1137/S0363012999357707
[3] Bobrovsky, B. Z. and Zakai, M. (1982). Asymptotic a priori estimates for the error in the nonlinear filtering problem. IEEE Trans. Inform. Theory 28 371–376.
Mathematical Reviews (MathSciNet):
MR651641
Digital Object Identifier: doi:10.1109/TIT.1982.1056460
[4] Budhiraja, A. (2003). Asymptotic stability, ergodicity and other asymptotic properties of the nonlinear filter. Ann. Inst. H. Poincaré Probab. Statist. 39 919–941.
Mathematical Reviews (MathSciNet):
MR2010391
Zentralblatt MATH:
1160.60318
Digital Object Identifier: doi:10.1016/S0246-0203(03)00022-0
[5] Budhiraja, A. and Kushner, H. J. (1999). Approximation and limit results for nonlinear filters over an infinite time interval. SIAM J. Control Optim. 37 1946–1979.
Mathematical Reviews (MathSciNet):
MR1720146
Digital Object Identifier: doi:10.1137/S0363012997328178
[6] Budhiraja, A. and Kushner, H. J. (2001). Monte Carlo algorithms and asymptotic problems in nonlinear filtering. In Stochastics in Finite and Infinite Dimensions 59–87. Birkhäuser, Boston.
Mathematical Reviews (MathSciNet):
MR1797081
Zentralblatt MATH:
1129.93535
[7] Cappé, O., Moulines, E. and Rydén, T. (2005). Inference in Hidden Markov Models. Springer, New York.
Mathematical Reviews (MathSciNet):
MR2159833
[8] Chigansky, P. (2009). Maximum likelihood estimator for hidden Markov models in continuous time. Stat. Inference Stoch. Process. 12 139–163. DOI 10.1007/s11203-008-9025-4.
Mathematical Reviews (MathSciNet):
MR2511676
Digital Object Identifier: doi:10.1007/s11203-008-9025-4
[9] Clark, J. M. C., Ocone, D. L. and Coumarbatch, C. (1999). Relative entropy and error bounds for filtering of Markov processes. Math. Control Signals Systems 12 346–360.
Mathematical Reviews (MathSciNet):
MR1728373
Zentralblatt MATH:
0940.93072
Digital Object Identifier: doi:10.1007/PL00009856
[10] Cogburn, R. (1984). The ergodic theory of Markov chains in random environments. Z. Wahrsch. Verw. Gebiete 66 109–128.
Mathematical Reviews (MathSciNet):
MR743088
[11] Cogburn, R. (1990). On direct convergence and periodicity for transition probabilities of Markov chains in random environments. Ann. Probab. 18 642–654.
Mathematical Reviews (MathSciNet):
MR1055425
Zentralblatt MATH:
0707.60057
Digital Object Identifier: doi:10.1214/aop/1176990850
Project Euclid: euclid.aop/1176990850
[12] Crisan, D. and Rozovsky, B. (2009). The Oxford University Handbook of Nonlinear Filtering. Oxford Univ. Press, Oxford.
[13] Del Moral, P. (1998). A uniform convergence theorem for the numerical solving of the nonlinear filtering problem. J. Appl. Probab. 35 873–884.
Mathematical Reviews (MathSciNet):
MR1671237
Zentralblatt MATH:
0940.60060
Digital Object Identifier: doi:10.1239/jap/1032438382
Project Euclid: euclid.jap/1032438382
[14] Del Moral, P. (2004). Feynman–Kac Formulae: Genealogical and Interacting Particle Systems With Applications. Springer, New York.
Mathematical Reviews (MathSciNet):
MR2044973
[15] Dellacherie, C. and Meyer, P.-A. (1982). Probabilities and Potential. B. North-Holland Mathematics Studies 72. North-Holland, Amsterdam.
Mathematical Reviews (MathSciNet):
MR745449
Zentralblatt MATH:
0494.60002
[16] Derriennic, Y. (1976). Lois “zéro ou deux” pour les processus de Markov. Applications aux marches aléatoires. Ann. Inst. H. Poincaré Sect. B (N.S.) 12 111–129.
Mathematical Reviews (MathSciNet):
MR423532
[17] Di Masi, G. B. and Stettner, Ł. (2005). Ergodicity of hidden Markov models. Math. Control Signals Systems 17 269–296.
Mathematical Reviews (MathSciNet):
MR2177319
Zentralblatt MATH:
1098.93036
Digital Object Identifier: doi:10.1007/s00498-005-0153-8
[18] Di Masi, G. B. and Stettner, Ł. (2008). Ergodicity of filtering process by vanishing discount approach. Systems Control Lett. 57 150–157.
Mathematical Reviews (MathSciNet):
MR2378760
[19] Douc, R. and Matias, C. (2001). Asymptotics of the maximum likelihood estimator for general hidden Markov models. Bernoulli 7 381–420.
Mathematical Reviews (MathSciNet):
MR1836737
Digital Object Identifier: doi:10.2307/3318493
Project Euclid: euclid.bj/1080004757
[20] Douc, R., Moulines, E. and Ritov, Y. (2009). Forgetting of the initial condition for the filter in general state-space hidden Markov chain: A coupling approach. Electron. J. Probab. 14 27–49.
Mathematical Reviews (MathSciNet):
MR2471658
[21] Kallenberg, O. (1997). Foundations of Modern Probability. Springer, New York.
Mathematical Reviews (MathSciNet):
MR1464694
[22] Karandikar, R. L. (1995). On pathwise stochastic integration. Stochastic Process. Appl. 57 11–18.
Mathematical Reviews (MathSciNet):
MR1327950
Zentralblatt MATH:
0816.60047
Digital Object Identifier: doi:10.1016/0304-4149(95)00002-O
[23] Kunita, H. (1971). Asymptotic behavior of the nonlinear filtering errors of Markov processes. J. Multivariate Anal. 1 365–393.
Mathematical Reviews (MathSciNet):
MR301812
Digital Object Identifier: doi:10.1016/0047-259X(71)90015-7
[24] Lindvall, T. (1992). Lectures on the Coupling Method. Wiley, New York.
Mathematical Reviews (MathSciNet):
MR1180522
[25] Ocone, D. and Pardoux, E. (1996). Asymptotic stability of the optimal filter with respect to its initial condition. SIAM J. Control Optim. 34 226–243.
Mathematical Reviews (MathSciNet):
MR1372912
Zentralblatt MATH:
1035.93508
Digital Object Identifier: doi:10.1137/S0363012993256617
[26] Orey, S. (1991). Markov chains with stochastically stationary transition probabilities. Ann. Probab. 19 907–928.
Mathematical Reviews (MathSciNet):
MR1112400
Zentralblatt MATH:
0735.60040
Digital Object Identifier: doi:10.1214/aop/1176990328
Project Euclid: euclid.aop/1176990328
[27] Orstein, D. and Sucheston, L. (1970). An operator theorem on L1 convergence to zero with applications to Markov kernels. Ann. Math. Statist. 41 1631–1639.
Mathematical Reviews (MathSciNet):
MR272057
Digital Object Identifier: doi:10.1214/aoms/1177696806
Project Euclid: euclid.aoms/1177696806
[28] Papangelou, F. (1976/77). A martingale approach to the convergence of the iterates of a transition function. Z. Wahrsch. Verw. Gebiete 37 211–226.
Mathematical Reviews (MathSciNet):
MR428450
Digital Object Identifier: doi:10.1007/BF00537489
[29] Revuz, D. (1975). Markov Chains. North-Holland, Amsterdam.
Mathematical Reviews (MathSciNet):
MR758799
[30] Revuz, D. (1983). Lois du tout ou rien et comportement asymptotique pour les probabilités de transition des processus de Markov. Ann. Inst. H. Poincaré Sect. B (N.S.) 19 9–24.
Mathematical Reviews (MathSciNet):
MR699975
[31] Runggaldier, W. J. and Stettner, Ł. (1993). Nearly optimal controls for stochastic ergodic problems with partial observation. SIAM J. Control Optim. 31 180–218.
Mathematical Reviews (MathSciNet):
MR1200230
Zentralblatt MATH:
0770.93092
Digital Object Identifier: doi:10.1137/0331013
[32] Stettner, Ł. (1989). On invariant measures of filtering processes. In Stochastic Differential Systems (Bad Honnef, 1988). Lecture Notes in Control and Information Sciences 126 279–292. Springer, Berlin.
Mathematical Reviews (MathSciNet):
MR1236074
Zentralblatt MATH:
0683.93082
Digital Object Identifier: doi:10.1007/BFb0043792
[33] Stratonovič, R. L. (1960). Conditional Markov processes. Teor. Verojatnost. i Primenen. 5 172–195.
Mathematical Reviews (MathSciNet):
MR137157
[34] Totoki, H. (1970). On a class of special flows. Z. Wahrsch. Verw. Gebiete 15 157–167.
Mathematical Reviews (MathSciNet):
MR279279
Digital Object Identifier: doi:10.1007/BF00531884
[35] van Handel, R. (2007). Filtering, stability, and robustness. Ph.D. thesis, California Institute of Technology.
[36] van Handel, R. (2009). Observability and nonlinear filtering. Probab. Theory Related Fields To appear.
Mathematical Reviews (MathSciNet):
MR2520121
Zentralblatt MATH:
1169.93025
Digital Object Identifier: doi:10.1007/s00440-008-0161-y
[37] von Weizsäcker, H. (1983). Exchanging the order of taking suprema and countable intersections of σ-algebras. Ann. Inst. H. Poincaré Sect. B (N.S.) 19 91–100.
Mathematical Reviews (MathSciNet):
MR699981
[38] von Weizsäcker, H. and Winkler, G. (1990). Stochastic Integrals. Friedr. Vieweg & Sohn, Braunschweig.
Mathematical Reviews (MathSciNet):
MR1062600
[39] Yor, M. (1977). Sur les théories du filtrage et de la prédiction. In Séminaire de Probabilités, XI (Univ. Strasbourg, Strasbourg, 1975/1976). Lecture Notes in Mathematics 581 257–297. Springer, Berlin.
Mathematical Reviews (MathSciNet):
MR471060
The Annals of Probability
