December 2013 On the embedding problem for discrete-time Markov chains
Marie-Anne Guerry
Author Affiliations +
J. Appl. Probab. 50(4): 918-930 (December 2013). DOI: 10.1239/jap/1389370090


When a discrete-time homogenous Markov chain is observed at time intervals that correspond to its time unit, then the transition probabilities of the chain can be estimated using known maximum likelihood estimators. In this paper we consider a situation when a Markov chain is observed on time intervals with length equal to twice the time unit of the Markov chain. The issue then arises of characterizing probability matrices whose square root(s) are also probability matrices. This characterization is referred to in the literature as the embedding problem for discrete time Markov chains. The probability matrix which has probability root(s) is called embeddable. In this paper for two-state Markov chains, necessary and sufficient conditions for embeddability are formulated and the probability square roots of the transition matrix are presented in analytic form. In finding conditions for the existence of probability square roots for (k x k) transition matrices, properties of row-normalized matrices are examined. Besides the existence of probability square roots, the uniqueness of these solutions is discussed: In the case of nonuniqueness, a procedure is introduced to identify a transition matrix that takes into account the specificity of the concrete context. In the case of nonexistence of a probability root, the concept of an approximate probability root is introduced as a solution of an optimization problem related to approximate nonnegative matrix factorization.


Download Citation

Marie-Anne Guerry. "On the embedding problem for discrete-time Markov chains." J. Appl. Probab. 50 (4) 918 - 930, December 2013.


Published: December 2013
First available in Project Euclid: 10 January 2014

zbMATH: 1306.60104
MathSciNet: MR3161364
Digital Object Identifier: 10.1239/jap/1389370090

Primary: 15A23 , 15B51 , 60J10

Keywords: embedding problem , Markov chain , probability matrix , root of a matrix

Rights: Copyright © 2013 Applied Probability Trust


This article is only available to subscribers.
It is not available for individual sale.

Vol.50 • No. 4 • December 2013
Back to Top