The Annals of Probability

The stability of conditional Markov processes and Markov chains in random environments

Ramon van Handel

Full-text: Open access

Abstract

We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this conditional signal is weakly ergodic when the signal is ergodic and the observations are nondegenerate. This permits a delicate exchange of the intersection and supremum of σ-fields, which is key for the stability of the nonlinear filter and partially resolves a long-standing gap in the proof of a result of Kunita [J. Multivariate Anal. 1 (1971) 365–393]. A similar result is obtained also in the continuous time setting. The proofs are based on an ergodic theorem for Markov chains in random environments in a general state space.

Article information

Source
Ann. Probab. Volume 37, Number 5 (2009), 1876-1925.

Dates
First available in Project Euclid: 21 September 2009

Permanent link to this document
https://projecteuclid.org/euclid.aop/1253539859

Digital Object Identifier
doi:10.1214/08-AOP448

Mathematical Reviews number (MathSciNet)
MR2561436

Zentralblatt MATH identifier
1178.93142

Subjects
Primary: 93E11: Filtering [See also 60G35]
Secondary: 60J05: Discrete-time Markov processes on general state spaces 62M20: Prediction [See also 60G25]; filtering [See also 60G35, 93E10, 93E11] 93E15: Stochastic stability

Keywords
Nonlinear filtering asymptotic stability hidden Markov models weak ergodicity tail σ-field exchange of intersection and supremum Markov chain in random environment

Citation

van Handel, Ramon. The stability of conditional Markov processes and Markov chains in random environments. Ann. Probab. 37 (2009), no. 5, 1876--1925. doi:10.1214/08-AOP448. https://projecteuclid.org/euclid.aop/1253539859


Export citation

References

  • [1] Baxendale, P., Chigansky, P. and Liptser, R. (2004). Asymptotic stability of the Wonham filter: Ergodic and nonergodic signals. SIAM J. Control Optim. 43 643–669.
  • [2] Bhatt, A. G., Budhiraja, A. and Karandikar, R. L. (2000). Markov property and ergodicity of the nonlinear filter. SIAM J. Control Optim. 39 928–949.
  • [3] Bobrovsky, B. Z. and Zakai, M. (1982). Asymptotic a priori estimates for the error in the nonlinear filtering problem. IEEE Trans. Inform. Theory 28 371–376.
  • [4] Budhiraja, A. (2003). Asymptotic stability, ergodicity and other asymptotic properties of the nonlinear filter. Ann. Inst. H. Poincaré Probab. Statist. 39 919–941.
  • [5] Budhiraja, A. and Kushner, H. J. (1999). Approximation and limit results for nonlinear filters over an infinite time interval. SIAM J. Control Optim. 37 1946–1979.
  • [6] Budhiraja, A. and Kushner, H. J. (2001). Monte Carlo algorithms and asymptotic problems in nonlinear filtering. In Stochastics in Finite and Infinite Dimensions 59–87. Birkhäuser, Boston.
  • [7] Cappé, O., Moulines, E. and Rydén, T. (2005). Inference in Hidden Markov Models. Springer, New York.
  • [8] Chigansky, P. (2009). Maximum likelihood estimator for hidden Markov models in continuous time. Stat. Inference Stoch. Process. 12 139–163. DOI 10.1007/s11203-008-9025-4.
  • [9] Clark, J. M. C., Ocone, D. L. and Coumarbatch, C. (1999). Relative entropy and error bounds for filtering of Markov processes. Math. Control Signals Systems 12 346–360.
  • [10] Cogburn, R. (1984). The ergodic theory of Markov chains in random environments. Z. Wahrsch. Verw. Gebiete 66 109–128.
  • [11] Cogburn, R. (1990). On direct convergence and periodicity for transition probabilities of Markov chains in random environments. Ann. Probab. 18 642–654.
  • [12] Crisan, D. and Rozovsky, B. (2009). The Oxford University Handbook of Nonlinear Filtering. Oxford Univ. Press, Oxford.
  • [13] Del Moral, P. (1998). A uniform convergence theorem for the numerical solving of the nonlinear filtering problem. J. Appl. Probab. 35 873–884.
  • [14] Del Moral, P. (2004). Feynman–Kac Formulae: Genealogical and Interacting Particle Systems With Applications. Springer, New York.
  • [15] Dellacherie, C. and Meyer, P.-A. (1982). Probabilities and Potential. B. North-Holland Mathematics Studies 72. North-Holland, Amsterdam.
  • [16] Derriennic, Y. (1976). Lois “zéro ou deux” pour les processus de Markov. Applications aux marches aléatoires. Ann. Inst. H. Poincaré Sect. B (N.S.) 12 111–129.
  • [17] Di Masi, G. B. and Stettner, Ł. (2005). Ergodicity of hidden Markov models. Math. Control Signals Systems 17 269–296.
  • [18] Di Masi, G. B. and Stettner, Ł. (2008). Ergodicity of filtering process by vanishing discount approach. Systems Control Lett. 57 150–157.
  • [19] Douc, R. and Matias, C. (2001). Asymptotics of the maximum likelihood estimator for general hidden Markov models. Bernoulli 7 381–420.
  • [20] Douc, R., Moulines, E. and Ritov, Y. (2009). Forgetting of the initial condition for the filter in general state-space hidden Markov chain: A coupling approach. Electron. J. Probab. 14 27–49.
  • [21] Kallenberg, O. (1997). Foundations of Modern Probability. Springer, New York.
  • [22] Karandikar, R. L. (1995). On pathwise stochastic integration. Stochastic Process. Appl. 57 11–18.
  • [23] Kunita, H. (1971). Asymptotic behavior of the nonlinear filtering errors of Markov processes. J. Multivariate Anal. 1 365–393.
  • [24] Lindvall, T. (1992). Lectures on the Coupling Method. Wiley, New York.
  • [25] Ocone, D. and Pardoux, E. (1996). Asymptotic stability of the optimal filter with respect to its initial condition. SIAM J. Control Optim. 34 226–243.
  • [26] Orey, S. (1991). Markov chains with stochastically stationary transition probabilities. Ann. Probab. 19 907–928.
  • [27] Orstein, D. and Sucheston, L. (1970). An operator theorem on L1 convergence to zero with applications to Markov kernels. Ann. Math. Statist. 41 1631–1639.
  • [28] Papangelou, F. (1976/77). A martingale approach to the convergence of the iterates of a transition function. Z. Wahrsch. Verw. Gebiete 37 211–226.
  • [29] Revuz, D. (1975). Markov Chains. North-Holland, Amsterdam.
  • [30] Revuz, D. (1983). Lois du tout ou rien et comportement asymptotique pour les probabilités de transition des processus de Markov. Ann. Inst. H. Poincaré Sect. B (N.S.) 19 9–24.
  • [31] Runggaldier, W. J. and Stettner, Ł. (1993). Nearly optimal controls for stochastic ergodic problems with partial observation. SIAM J. Control Optim. 31 180–218.
  • [32] Stettner, Ł. (1989). On invariant measures of filtering processes. In Stochastic Differential Systems (Bad Honnef, 1988). Lecture Notes in Control and Information Sciences 126 279–292. Springer, Berlin.
  • [33] Stratonovič, R. L. (1960). Conditional Markov processes. Teor. Verojatnost. i Primenen. 5 172–195.
  • [34] Totoki, H. (1970). On a class of special flows. Z. Wahrsch. Verw. Gebiete 15 157–167.
  • [35] van Handel, R. (2007). Filtering, stability, and robustness. Ph.D. thesis, California Institute of Technology.
  • [36] van Handel, R. (2009). Observability and nonlinear filtering. Probab. Theory Related Fields To appear.
  • [37] von Weizsäcker, H. (1983). Exchanging the order of taking suprema and countable intersections of σ-algebras. Ann. Inst. H. Poincaré Sect. B (N.S.) 19 91–100.
  • [38] von Weizsäcker, H. and Winkler, G. (1990). Stochastic Integrals. Friedr. Vieweg & Sohn, Braunschweig.
  • [39] Yor, M. (1977). Sur les théories du filtrage et de la prédiction. In Séminaire de Probabilités, XI (Univ. Strasbourg, Strasbourg, 1975/1976). Lecture Notes in Mathematics 581 257–297. Springer, Berlin.