Advances in Applied Probability

Markov-modulated Ornstein-Uhlenbeck processes

G. Huang, H. M. Jansen, M. Mandjes, P. Spreij, and K. De Turck

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


In this paper we consider an Ornstein-Uhlenbeck (OU) process (M(t))t≥0 whose parameters are determined by an external Markov process (X(t))t≥0 on a finite state space {1, . . ., d}; this process is usually referred to as Markov-modulated Ornstein-Uhlenbeck. We use stochastic integration theory to determine explicit expressions for the mean and variance of M(t). Then we establish a system of partial differential equations (PDEs) for the Laplace transform of M(t) and the state X(t) of the background process, jointly for time epochs t = t1, . . ., tK. Then we use this PDE to set up a recursion that yields all moments of M(t) and its stationary counterpart; we also find an expression for the covariance between M(t) and M(t + u). We then establish a functional central limit theorem for M(t) for the situation that certain parameters of the underlying OU processes are scaled, in combination with the modulating Markov process being accelerated; interestingly, specific scalings lead to drastically different limiting processes. We conclude the paper by considering the situation of a single Markov process modulating multiple OU processes.

Article information

Adv. in Appl. Probab., Volume 48, Number 1 (2016), 235-254.

First available in Project Euclid: 8 March 2016

Permanent link to this document

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 60K25: Queueing theory [See also 68M20, 90B22]
Secondary: 60G44: Martingales with continuous parameter 60G15: Gaussian processes

Ornstein-Uhlenbeck process Markov modulation regime switching central limit theorems martingale techniques


Huang, G.; Jansen, H. M.; Mandjes, M.; Spreij, P.; De Turck, K. Markov-modulated Ornstein-Uhlenbeck processes. Adv. in Appl. Probab. 48 (2016), no. 1, 235--254.

Export citation


  • Alili, L., Patie, P. and Pedersen, J. L. (2005). Representations of the first hitting time density of an Ornstein–Uhlenbeck process. Stoch. Models 21, 967–980.
  • Anderson, D., Blom, J., Mandjes, M., Thorsdottir, H. and De Turck, K. (2014). A functional central limit theorem for a Markov-modulated infinite-server queue. To appear in Methodology Comput. Appl. Prob..
  • Ang, A. and Bekaert, G. (2002). Regime switches in interest rates. J. Business Econom. Statist. 20, 163–182.
  • Asmussen, S. (2003). Applied Probability and Queues, 2nd edn. Springer, New York.
  • Asmussen, S. and Albrecher, H. (2010). Ruin Probabilities, 2nd edn. World Scientific, Hackensack, NJ.
  • Banachewicz, K., Lucas, A. and van der Vaart, A. (2008). Modelling portfolio defaults using hidden Markov models with covariates. Econometrics J. 11, 155–171.
  • Blom, J. and Mandjes, M. (2013). A large-deviations analysis of Markov-modulated inifinite-server queues. Operat. Res. Lett. 41, 220–225.
  • Blom, J., De Turck, K. and Mandjes, M. (2013). Rare event analysis of Markov-modulated infinite-server queues: a Poisson limit. Stoch. Models 29, 463–474.
  • Blom, J., De Turck, K. and Mandjes, M. (2015). Analysis of Markov-modulated infinite-server queues in the central-limit regime. Prob. Eng. Inf. Sci. 29, 433–459.
  • Blom, J., Mandjes, M. and Thorsdottir, H. (2013). Time-scaling limits for Markov-modulated infinite-server queues. Stoch. Models 29, 112–127.
  • Blom, J., De Turck, K., Kella, O. and Mandjes, M. (2014). Tail asymptotics of a Markov-modulated infinite-server queue. Queueing Systems 78, 337–357.
  • Blom, J., Kella, O., Mandjes, M. and Thorsdottir, H. (2014). Markov-modulated infinite-server queues with general service times. Queueing Systems 76, 403–424.
  • Coolen-Schrijner, P. and van Doorn, E. A. (2002). The deviation matrix of a continuous-time Markov chain. Prob. Eng. Inf. Sci. 16, 351–366.
  • D'Auria, B. (2008). M/M/$\infty$ queues in semi-Markovian random environment. Queueing Systems 58, 221–237.
  • Elliott, R. J. and Mamon, R. S. (2002). An interest rate model with a Markovian mean reverting level. Quant. Finance 2, 454–458.
  • Elliott, R. J. and Siu, T. K. (2009). On Markov-modulated exponential-affine bond price formulae. Appl. Math. Finance 16, 1–15.
  • Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes. Characterization and Convergence. John Wiley, New York.
  • Fralix, B. H. and Adan, I. J. B. F. (2009). An infinite-server queue influenced by a semi-Markovian environment. Queueing Systems 61, 65–84.
  • Giampieri, G., Davis, M. and Crowder, M. (2005). Analysis of default data using hidden Markov models. Quant. Finance 5, 27–34.
  • Guyon, X., Iovleff, S. and Yao, J.-F. (2004). Linear diffusion with stationary switching regime. ESAIM Prob. Statist. 8, 25–35.
  • Hale, J. K. (1980). Ordinary Differential Equations, 2nd edn. Krieger, Huntington, New York.
  • Hamilton, J. D. (1989). A new approach to the economic analysis of nonstationary time series and the business cycle. Econometrica 57, 357–384.
  • Horn, R. A. and Johnson, C. R. (1985). Matrix Analysis. Cambridge University Press.
  • Huang, G., Mandjes, M. and Spreij, P. (2014). Weak convergence of Markov-modulated diffusion processes with rapid switching. Statist. Prob. Lett. 86, 74–79.
  • Huang, G. \et (2014). Markov-modulated Ornstein–Uhlenbeck processes. Preprint. Available at http://arxiv. org/abs/1412.7952v1.
  • Jacobsen, M. (1996). Laplace and the origin of the Ornstein–Uhlenbeck process. Bernoulli 2, 271–286.
  • Jacod, J. and Shiryaev, A. N. (2003). Limit Theorems for Stochastic Processes, 2nd edn. Springer, Berlin.
  • Karatzas, I. and Shreve, S. E. (1991). Brownian Motion and Stochastic Calculus, 2nd edn. Springer, New York.
  • Keilson, J. (1979). Markov Chain Models–-Rarity and Exponentiality. Springer, New York.
  • Kemeny, J. G. and Snell, J. L. (1960). Finite Markov Chains. Van Nostrand, Princeton, NJ.
  • Neuts, M. F. (1981). Matrix–Geometric Solutions in Stochastic Models: An Algorithmic Approach. John Hopkins University Press, Baltimore, MD.
  • O'Cinneide, C. A. and Purdue, P. (1986). The M/M/$\infty$ queue in a random environment. J. Appl. Prob. 23, 175–184.
  • Revuz, D. and Yor, M. (1999). Continuous Martingales and Brownian Motion, 3rd edn. Springer, Berlin.
  • Robert, P. (2003). Stochastic Networks and Queues. Springer, Berlin.
  • Rogers, L. C. G. and Williams, D. (2000). Diffusions, Markov Processes, and Martingales, Vol. 2, Itô Calculus, 2nd edn. Cambridge University Press.
  • Syski, R. (1978). Ergodic potential. Stoch. Process. Appl. 7, 311–336.
  • Uhlenbeck, G. E. and Ornstein, L. S. (1930). On the theory of Brownian motion. Phys. Rev. 36, 823–841.
  • Williams, D. (1991). Probability with Martingales. Cambridge University Press.
  • Xing, X., Zhang, W. and Wang, Y. (2009). The stationary distributions of two classes of reflected Ornstein–Uhlenbeck processes. J. Appl. Prob. 46, 709–720.
  • Yuan, C. and Mao, X. (2003). Asymptotic stability in distribution of stochastic differential equations with Markovian switching. Stoch. Process. Appl. 103, 277–291.