A renewal theory is developed for sums of independent random variables whose distributions are determined by the current state of a Markov chain (also known as "Markov additive" processes, or "semi-Markov" processes). This theory departs from existing theories in that its conclusions are required to be valid conditionally for a given realization of the Markov Chain. It rests on a peculiar coupling construction which differs markedly from existing coupling arguments.
"Conditional Markov Renewal Theory I. Finite and Denumerable State Space." Ann. Probab. 12 (4) 1113 - 1148, November, 1984. https://doi.org/10.1214/aop/1176993144