Bernoulli

  • Bernoulli
  • Volume 24, Number 3 (2018), 1942-1972.

Tree formulas, mean first passage times and Kemeny’s constant of a Markov chain

Jim Pitman and Wenpin Tang

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

This paper offers some probabilistic and combinatorial insights into tree formulas for the Green function and hitting probabilities of Markov chains on a finite state space. These tree formulas are closely related to loop-erased random walks by Wilson’s algorithm for random spanning trees, and to mixing times by the Markov chain tree theorem. Let $m_{ij}$ be the mean first passage time from $i$ to $j$ for an irreducible chain with finite state space $S$ and transition matrix $(p_{ij};i,j\in S)$. It is well known that $m_{jj}=1/\pi_{j}=\Sigma^{(1)}/\Sigma_{j}$, where $\pi$ is the stationary distribution for the chain, $\Sigma_{j}$ is the tree sum, over $n^{n-2}$ trees $\mathbf{t}$ spanning $S$ with root $j$ and edges $i\rightarrow k$ directed towards $j$, of the tree product $\prod_{i\rightarrow k\in\mathbf{t}}p_{ik}$, and $\Sigma^{(1)}:=\sum_{j\in S}\Sigma_{j}$. Chebotarev and Agaev (Linear Algebra Appl. 356 (2002) 253–274) derived further results from Kirchhoff’s matrix tree theorem. We deduce that for $i\ne j$, $m_{ij}=\Sigma_{ij}/\Sigma_{j}$, where $\Sigma_{ij}$ is the sum over the same set of $n^{n-2}$ spanning trees of the same tree product as for $\Sigma_{j}$, except that in each product the factor $p_{kj}$ is omitted where $k=k(i,j,\mathbf{t})$ is the last state before $j$ in the path from $i$ to $j$ in $\mathbf{t}$. It follows that Kemeny’s constant $\sum_{j\in S}m_{ij}/m_{jj}$ equals $\Sigma^{(2)}/\Sigma^{(1)}$, where $\Sigma^{(r)}$ is the sum, over all forests $\mathbf{f}$ labeled by $S$ with $r$ directed trees, of the product of $p_{ij}$ over edges $i\rightarrow j$ of $\mathbf{f}$. We show that these results can be derived without appeal to the matrix tree theorem. A list of relevant literature is also reviewed.

Article information

Source
Bernoulli Volume 24, Number 3 (2018), 1942-1972.

Dates
Received: May 2016
Revised: September 2016
First available in Project Euclid: 2 February 2018

Permanent link to this document
https://projecteuclid.org/euclid.bj/1517540464

Digital Object Identifier
doi:10.3150/16-BEJ916

Keywords
Cayley’s formula Green tree formula harmonic tree formula Kemeny’s constant Kirchhoff’s matrix tree theorem Markov chain tree theorem mean first passage times spanning forests/trees Wilson’s algorithm

Citation

Pitman, Jim; Tang, Wenpin. Tree formulas, mean first passage times and Kemeny’s constant of a Markov chain. Bernoulli 24 (2018), no. 3, 1942--1972. doi:10.3150/16-BEJ916. https://projecteuclid.org/euclid.bj/1517540464


Export citation

References

  • [1] Abdesselam, A. (2004). The Grassmann–Berezin calculus and theorems of the matrix-tree type. Adv. in Appl. Math. 33 51–70.
  • [2] Aldous, D. and Fill, J.A. (2002). Markov Chains and random walks on graphs. Available at http://www.stat.berkeley.edu/aldous/RWG/book.html.
  • [3] Aldous, D.J. (1990). The random walk construction of uniform spanning trees and uniform labelled trees. SIAM J. Discrete Math. 3 450–465.
  • [4] Anantharam, V. and Tsoucas, P. (1989). A proof of the Markov chain tree theorem. Statist. Probab. Lett. 8 189–192.
  • [5] Avena, L. and Gaudilliere, A. (2013). On some random forests with determinantal roots. Preprint. Available at arXiv:1310.1723.
  • [6] Bellman, R. (1997). Introduction to Matrix Analysis. Classics in Applied Mathematics 19. Philadelphia, PA: SIAM.
  • [7] Benjamin, A.T. and Yerger, C.R. (2006). Combinatorial interpretations of spanning tree identities. Bull. Inst. Combin. Appl. 47 37–42.
  • [8] Benjamini, I., Lyons, R., Peres, Y. and Schramm, O. (2001). Uniform spanning forests. Ann. Probab. 29 1–65.
  • [9] Benjamini, I. and Schramm, O. (2001). Recurrence of distributional limits of finite planar graphs. Electron. J. Probab. 6 no. 23, 13 pp. (electronic).
  • [10] Biane, P. (2015). Polynomials associated with finite Markov chains. In In Memoriam Marc Yor – Séminaire de Probabilités XLVII. Lecture Notes in Math. 2137 249–262. Cham: Springer.
  • [11] Biane, P. and Chapuy, G. (2015). Laplacian matrices and spanning trees of tree graphs. Preprint. Available at arXiv:1505.04806.
  • [12] Biggs, N. (1993). Algebraic Graph Theory, 2nd ed. Cambridge Mathematical Library. Cambridge: Cambridge Univ. Press.
  • [13] Bǫcher, M. (1964). Introduction to Higher Algebra. New York: Dover Publications.
  • [14] Boesch, F.T. and Prodinger, H. (1986). Spanning tree formulas and Chebyshev polynomials. Graphs Combin. 2 191–200.
  • [15] Bogner, C. and Weinzierl, S. (2010). Feynman graph polynomials. Internat. J. Modern Phys. A 25 2585–2618.
  • [16] Bott, R. and Mayberry, J.P. (1954). Matrices and trees. In Economic Activity Analysis 391–400. New York: Wiley.
  • [17] Broder, A. (1989). Generating random spanning trees. In Proceedings of the 30th Annual Symposium on Foundations of Computer Science (SFCS’89) 442–447. Washington, DC: IEEE Computer Society.
  • [18] Brooks, R.L., Smith, C.A.B., Stone, A.H. and Tutte, W.T. (1940). The dissection of rectangles into squares. Duke Math. J. 7 312–340.
  • [19] Burton, R. and Pemantle, R. (1993). Local characteristics, entropy and limit theorems for spanning trees and domino tilings via transfer-impedances. Ann. Probab. 21 1329–1371.
  • [20] Catral, M., Kirkland, S.J., Neumann, M. and Sze, N.-S. (2010). The Kemeny constant for finite homogeneous ergodic Markov chains. J. Sci. Comput. 45 151–166.
  • [21] Cayley, A. (1889). A theorem on trees. Pure Appl. Math. Q. 23 376–378.
  • [22] Chaiken, S. (1982). A combinatorial proof of the all minors matrix tree theorem. SIAM J. Algebr. Discrete Methods 3 319–329.
  • [23] Chaiken, S. and Kleitman, D.J. (1978). Matrix tree theorems. J. Combin. Theory Ser. A 24 377–381.
  • [24] Chang, Y. and Jan, Y.L. (2014). Markov loops in discrete spaces. Preprint. Available at arXiv:1402.1064.
  • [25] Chebotarev, P. (2007). A graph theoretic interpretation of the mean first passage times. Preprint. Available at arXiv:Math/0701359.
  • [26] Chebotarev, P. and Agaev, R. (2002). Forest matrices around the Laplacian matrix. Linear Algebra Appl. 356 253–274.
  • [27] Chen, W.K. (1976). Applied Graph Theory, revised ed. Amsterdam: North-Holland.
  • [28] Chung, K.L. (1960). Markov Chains with Stationary Transition Probabilities. Die Grundlehren der Mathematischen Wissenschaften 104. Berlin: Springer.
  • [29] Cori, R. and Le Borgne, Y. (2003). The sand-pile model and Tutte polynomials. Adv. in Appl. Math. 30 44–52.
  • [30] de la Harpe, P. (2012). The Fuglede–Kadison determinant, theme and variations. Preprint. Available at arXiv:1107.1059.
  • [31] Dhar, D. (1990). Self-organized critical state of sandpile automaton models. Phys. Rev. Lett. 64 1613–1616.
  • [32] de Tilière, B. (2014). Principal minors Pfaffian half-tree theorem. J. Combin. Theory Ser. A 124 1–40.
  • [33] Doyle, P. (2009). The Kemeny constant of a Markov chain. Available at https://math.dartmouth.edu/~doyle/docs/kc/kc.pdf.
  • [34] Durrett, R. (2010). Probability: Theory and Examples, 4th ed. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge: Cambridge Univ. Press.
  • [35] Duval, A.M., Klivans, C.J. and Martin, J.L. (2009). Simplicial matrix-tree theorems. Trans. Amer. Math. Soc. 361 6073–6114.
  • [36] Feller, W. (1968). An Introduction to Probability Theory and Its Applications. Vol. I, 3rd ed. New York: Wiley.
  • [37] Forman, R. (1993). Determinants of Laplacians on graphs. Topology 32 35–46.
  • [38] Gorodezky, I. and Pak, I. (2014). Generalized loop-erased random walks and approximate reachability. Random Structures Algorithms 44 201–223.
  • [39] Grimmett, G. (2010). Probability on Graphs: Random Processes on Graphs and Lattices. Institute of Mathematical Statistics Textbooks 1. Cambridge: Cambridge Univ. Press.
  • [40] Gursoy, B.B., Kirkland, S., Mason, O. and Sergeev, S. (2013). On the Markov chain tree theorem in the max algebra. Electron. J. Linear Algebra 26 15–27.
  • [41] Gustafson, K. and Hunter, J.J. (2015). Why the Kemeny Time is a constant. Preprint. Available at arXiv:1510.00456.
  • [42] Hunter, J.J. (2014). The role of Kemeny’s constant in properties of Markov chains. Comm. Statist. Theory Methods 43 1309–1321.
  • [43] Hunter, J.J. (2016). Accurate calculations of stationary distributions and mean first passage times in Markov renewal processes and Markov chains. Spec. Matrices 4 151–175.
  • [44] Járai, A.A. (2014). Sandpile models. Preprint. Available at arXiv:1401.0354.
  • [45] Kalai, G. (1983). Enumeration of ${\mathbf{Q}}$-acyclic simplicial complexes. Israel J. Math. 45 337–351.
  • [46] Kassel, A. (2015). Learning about critical phenomena from scribbles and sandpiles. In Modélisation Aléatoire et Statistique – Journées MAS 2014. ESAIM Proc. Surveys 51 60–73. Les Ulis: EDP Sci.
  • [47] Kassel, A. and Kenyon, R. (2012). Random curves on surfaces induced from the Laplacian determinant. Preprint. Available at arXiv:1211.6974.
  • [48] Kassel, A., Kenyon, R. and Wu, W. (2015). Random two-component spanning forests. Ann. Inst. Henri Poincaré Probab. Stat. 51 1457–1464.
  • [49] Kassel, A. and Wilson, D.B. (2016). The looping rate and sandpile density of planar graphs. Amer. Math. Monthly 123 19–39.
  • [50] Kasteleyn, P.W. (1967). Graph theory and crystal physics. In Graph Theory and Theoretical Physics 43–110. London: Academic Press.
  • [51] Kelmans, A.K. and Chelnokov, V.M. (1974). A certain polynomial of a graph and graphs with an extremal number of trees. J. Combin. Theory Ser. B 16 197–214.
  • [52] Kelner, J.A. and Ma̧dry, A. (2009). Faster generation of random spanning trees. In 2009 50th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2009) 13–21. Los Alamitos, CA: IEEE Computer Soc.
  • [53] Kemeny, J.G. and Snell, J.L. (1960). Finite Markov Chains. The University Series in Undergraduate Mathematics. Princeton, NJ: D. Van Nostrand Co.
  • [54] Kenyon, R. (2009). Lectures on dimers. In Statistical Mechanics. IAS/Park City Math. Ser. 16 191–230. Providence, RI: Amer. Math. Soc.
  • [55] Kenyon, R. (2011). Spanning forests and the vector bundle Laplacian. Ann. Probab. 39 1983–2017.
  • [56] Kenyon, R.W., Propp, J.G. and Wilson, D.B. (2000). Trees and matchings. Electron. J. Combin. 7 Research Paper 25, 34 pp. (electronic).
  • [57] Kenyon, R.W. and Wilson, D.B. (2015). Spanning trees of graphs on surfaces and the intensity of loop-erased random walk on planar graphs. J. Amer. Math. Soc. 28 985–1030.
  • [58] Kirchhoff, G. (1847). Über die Auflösung der Gleichungen, auf welche man bei der Untersuchung der linearen Verteilung galvanischer Ströme geführt wird. Ann. Phys. Chem. 72 497–508.
  • [59] Kirchhoff, G. (1958). On the solution of the equations obtained from the investigation of the linear distribution of galvanic currents (translated by J.B. O’Toole). IRE Trans. Circuit Theory 5 4–7.
  • [60] Kohler, H.-H. and Vollmerhaus, E. (1980). The frequency of cyclic processes in biological multistate systems. J. Math. Biol. 9 275–290.
  • [61] Kozdron, M.J., Richards, L.M. and Stroock, D.W. (2013). Determinants, their applications to Markov processes, and a random walk proof of Kirchhoff’s matrix tree theorem. Preprint. Available at arXiv:1306.2059.
  • [62] Lawler, G.F. (1999). Loop-erased random walk. In Perplexing Problems in Probability. Progress in Probability 44 197–217. Boston, MA: Birkhäuser.
  • [63] Lawler, G.F. and Limic, V. (2010). Random Walk: A Modern Introduction. Cambridge Studies in Advanced Mathematics 123. Cambridge: Cambridge Univ. Press.
  • [64] Leighton, F.T. and Rivest, R.L. (1983). Estimating a probability using finite memory. In Foundations of Computation Theory (Borgholm, 1983). Lecture Notes in Computer Science 158 255–269. Berlin: Springer.
  • [65] Leighton, F.T. and Rivest, R.L. (1983). The Markov chain tree theorem. Technical Report, M.I.T Laboratory for Computer Science, MIT/LCS/TM-249.
  • [66] Levene, M. and Loizou, G. (2002). Kemeny’s constant and the random surfer. Amer. Math. Monthly 109 741–745.
  • [67] Levin, D.A., Peres, Y. and Wilmer, E.L. (2009). Markov Chains and Mixing Times. Providence, RI: Amer. Math. Soc.
  • [68] Liu, C.J. and Chow, Y. (1981). Enumeration of forests in a graph. Proc. Amer. Math. Soc. 83 659–662.
  • [69] Lovász, L. and Winkler, P. (1998). Mixing times. In Microsurveys in Discrete Probability (Princeton, NJ, 1997). DIMACS Ser. Discrete Math. Theoret. Comput. Sci. 41 85–133. Providence, RI: Amer. Math. Soc.
  • [70] Lyons, R. (1998). A bird’s-eye view of uniform spanning trees and forests. In Microsurveys in Discrete Probability (Princeton, NJ, 1997). DIMACS Ser. Discrete Math. Theoret. Comput. Sci. 41 135–162. Providence, RI: Amer. Math. Soc.
  • [71] Lyons, R. (2005). Asymptotic enumeration of spanning trees. Combin. Probab. Comput. 14 491–522.
  • [72] Lyons, R. (2009). Random complexes and $l^{2}$-Betti numbers. J. Topol. Anal. 1 153–175.
  • [73] Lyons, R. (2010). Identities and inequalities for tree entropy. Combin. Probab. Comput. 19 303–313.
  • [74] Lyons, R. and Peres, Y. (2016). Probability on Trees and Networks. Cambridge: Cambridge Univ. Press. Available at http://pages.iu.edu/~rdlyons/.
  • [75] Ma̧dry, A., Straszak, D. and Tarnawski, J. (2015). Fast generation of random spanning trees and the effective resistance metric. In Proceedings of the Twenty-Sixth Annual ACM-SIAM Symposium on Discrete Algorithms 2019–2036. Philadelphia, PA: SIAM.
  • [76] Majumdar, S.N. and Dhar, D. (1992). Equivalence between the Abelian sandpile model and the $q\rightarrow0$ limit of the Potts model. Physica A: Statistical Mechanics and Its Applications 185 129–145.
  • [77] Marchal, P. (1999). Cycles hamiltoniens aléatoires et mesures d’occupation invariantes par une action de groupe. C. R. Acad. Sci. Paris Sér. I Math. 329 883–886.
  • [78] Marchal, P. (2000). Loop-erased random walks, spanning trees and Hamiltonian cycles. Electron. Commun. Probab. 5 39–50 (electronic).
  • [79] Masbaum, G. and Vaintrob, A. (2002). A new matrix-tree theorem. Int. Math. Res. Not. 27 1397–1426.
  • [80] Meyer, C. (2000). Matrix Analysis and Applied Linear Algebra. Philadelphia, PA: SIAM.
  • [81] Minoux, M. (1997). Bideterminants, arborescences and extension of the matrix-tree theorem to semirings. Discrete Math. 171 191–200.
  • [82] Moon, J.W. (1970). Counting Labelled Trees. From Lectures Delivered to the Twelfth Biennial Seminar of the Canadian Mathematical Congress (Vancouver) 1969. Montreal, Que.: Canadian Mathematical Congress.
  • [83] Orlin, J.B. (1978). Line-digraphs, arborescences, and theorems of Tutte and Knuth. J. Combin. Theory Ser. B 25 187–198.
  • [84] Pak, I. and Postnikov, A. (1994). Enumeration of spanning trees of graphs. Available at http://math.mit.edu/~apost/papers/tree.ps.
  • [85] Pemantle, R. (1991). Choosing a spanning tree for the integer lattice uniformly. Ann. Probab. 19 1559–1574.
  • [86] Pitman, J. Enumerations and expectations related to the matrix tree expansion of a determinant. In preparation.
  • [87] Pitman, J. (2001/2002). Random mappings, forests, and subsets associated with Abel–Cayley–Hurwitz multinomial expansions. Sém. Lothar. Combin. 46 Art. B46h, 45 pp. (electronic).
  • [88] Pitman, J. (2002). Forest volume decompositions and Abel–Cayley–Hurwitz multinomial expansions. J. Combin. Theory Ser. A 98 175–191.
  • [89] Pitman, J.W. (1977). Occupation measures for Markov chains. Adv. in Appl. Probab. 9 69–86.
  • [90] Pokarowski, P. (1999). Directed forests with application to algorithms related to Markov chains. Appl. Math. (Warsaw) 26 395–414.
  • [91] Propp, J.G. and Wilson, D.B. (1998). How to get a perfectly random sample from a generic Markov chain and generate a random spanning tree of a directed graph. J. Algorithms 27 170–217.
  • [92] Runge, F. and Sachs, H. (1974). Berechnung der Anzahl der Gerüste von Graphen und Hypergraphen mittels deren Spektren. Math. Balkanica 4 529–536.
  • [93] Sahi, S. (2014). Harmonic vectors and matrix tree theorems. J. Comb. 5 195–202.
  • [94] Seneta, E. (2006). Non-negative Matrices and Markov Chains. Springer Series in Statistics. New York: Springer.
  • [95] Shubert, B.O. (1975). A flow-graph formula for the stationary distribution of a Markov chain. IEEE Trans. Syst. Man Cybern. Syst. SMC-5 565–566.
  • [96] Sokal, A.D. (2005). The multivariate Tutte polynomial (alias Potts model) for graphs and matroids. In Surveys in Combinatorics 2005. London Mathematical Society Lecture Note Series 327 173–226. Cambridge: Cambridge Univ. Press.
  • [97] Solberg, J.J. (1974/1975). A graph theoretic formula for the steady state distribution of finite Markov processes. Manage. Sci. 21 1040–1048.
  • [98] Stanley, R.P. (1999). Enumerative Combinatorics. Vol. 2. Cambridge Studies in Advanced Mathematics 62. Cambridge: Cambridge Univ. Press.
  • [99] Stroock, D.W. (2014). An Introduction to Markov Processes, 2nd ed. Graduate Texts in Mathematics 230. Heidelberg: Springer.
  • [100] Temperley, H.N.V. (1964). On the mutual cancellation of cluster integrals in Mayer’s fugacity series. Proc. Phys. Soc. 83 3–16.
  • [101] Temperley, H.N.V. (1972). The enumeration of graphs on large periodic lattices. In Combinatorics (Proc. Conf. Combinatorial Math., Math. Inst., Oxford, 1972) 285–294. Southend-on-Sea: Inst. Math. Appl.
  • [102] Temperley, H.N.V. (1974). Enumeration of graphs on a large periodic lattice. In Combinatorics (Proc. British Combinatorial Conf., Univ. Coll. Wales, Aberystwyth, 1973). London Math. Soc. Lecture Note Ser. 13 155–159. London: Cambridge Univ. Press.
  • [103] Temperley, H.N.V. and Fisher, M.E. (1961). Dimer problem in statistical mechanics – an exact result. Philos. Mag. (8) 6 1061–1063.
  • [104] Tutte, W.T. (1948). The dissection of equilateral triangles into equilateral triangles. Math. Proc. Cambridge Philos. Soc. 44 463–482.
  • [105] Tutte, W.T. (2001). Graph Theory. Encyclopedia of Mathematics and Its Applications 21. Cambridge: Cambridge Univ. Press.
  • [106] Ventcel, A.D. and Freĭdlin, M.I. (1970). Small random perturbations of dynamical systems. Uspekhi Mat. Nauk 25 3–55.
  • [107] Wilson, D.B. (1996). Generating random spanning trees more quickly than the cover time. In Proceedings of the Twenty-Eighth Annual ACM Symposium on the Theory of Computing (Philadelphia, PA, 1996) 296–303. New York: ACM.
  • [108] Wu, F.Y. (2002). Dimers and spanning trees: Some recent results. Internat. J. Modern Phys. B 16 1951–1961.
  • [109] Zeilberger, D. (1985). A combinatorial approach to matrix algebra. Discrete Math. 56 61–72.
  • [110] Zhang, Y., Yong, X. and Golin, M.J. (2005). Chebyshev polynomials and spanning tree formulas for circulant and related graphs. Discrete Math. 298 334–364.