Bounding Stationary Expectations of Markov Processes
Peter W. Glynn, Assaf Zeevi
Abstract
This paper develops a simple and systematic approach for obtaining bounds on stationary expectations of Markov processes. Given a function f which one is interested in evaluating, the main idea is to find a function g that satisfies a certain “mean drift” inequality with respect to f, which in turn leads to bounds on the stationary expectation of the latter. The approach developed in the paper is broadly applicable and can be used to bound steady-state expectations in general state space Markov chains, continuous time chains, and diffusion processes (with, or without, reflecting boundaries).
First Page:
Show
Hide
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.imsc/1233152943
Digital Object Identifier: doi:10.1214/074921708000000381
References
[1] Asmussen, S. (2003). Applied Probability and Queues. Springer, NY.
Mathematical Reviews (MathSciNet):
MR1978607
[2] Bertsimas, D., Gamarnik, D. and Tsitsiklis, J. (2001). Performance of multiclass Markovian queueing networks via piecewise linear Lyapunov functions. Ann. Appl. Probab. 11 1384–1428.
Mathematical Reviews (MathSciNet):
MR1878302
Zentralblatt MATH:
1012.60082
Digital Object Identifier: doi:10.1214/aoap/1015345407
Project Euclid: euclid.aoap/1015345407
[3] Bertsimas, D., Paschalidis, I. and Tsitsiklis, J. (1994). Optimization of multiclass queueing networks: Polyhedral and nonlinear characterizations of achievable performance. Ann. Appl. Probab. 4 43–75.
Mathematical Reviews (MathSciNet):
MR1258173
Zentralblatt MATH:
0797.60079
Digital Object Identifier: doi:10.1214/aoap/1177005200
Project Euclid: euclid.aoap/1177005200
[4] Borovkov, A. A. (2000). Ergodicity and Stability of Stochastic Processes. John Wiley & Sons, New York.
Mathematical Reviews (MathSciNet):
MR1658404
[5] Budhiraja, A. and Lee, C. (2007). Long time asymptotics for constrained diffusions in polyhedral domains. Stochastic Processes and Their Applications 117 1014–1036.
Mathematical Reviews (MathSciNet):
MR2340877
Zentralblatt MATH:
1119.60066
Digital Object Identifier: doi:10.1016/j.spa.2006.11.007
[6] Dai, J. and Harrison, J. M. (2008). Reflecting Brownian motion in the orthant: An illuminating example of instability. Math. of Oper. Res., to appear.
[7] Dupuis, P. and Williams, R. J. (1994). Lyapunov functions for semimartingale reflecting Brownian motions. Ann. Probab. 22 680–702.
Mathematical Reviews (MathSciNet):
MR1288127
Zentralblatt MATH:
0808.60068
Digital Object Identifier: doi:10.1214/aop/1176988725
Project Euclid: euclid.aop/1176988725
[8] Fayolle, G. (1989). On random walks arising in queueing systems: Ergodicity and transience via quadratic forms as Lyapunov functions. Part I. Queueing Systems 5 167–184.
Mathematical Reviews (MathSciNet):
MR1032553
Digital Object Identifier: doi:10.1007/BF01149191
[9] Gamarnik, D. and Zeevi, A. (2006). Validity of heavy-traffic steady-state approximations in generalized Jackson networks. Ann. Appl. Probab. 16 56–90.
Mathematical Reviews (MathSciNet):
MR2209336
Digital Object Identifier: doi:10.1214/105051605000000638
Project Euclid: euclid.aoap/1141654281
[10] Glynn, P. W. and Meyn, S. P. (1996). A Liapounov bound for solutions of the Poisson equation. Ann. Probab. 24 916–931.
Mathematical Reviews (MathSciNet):
MR1404536
Zentralblatt MATH:
0863.60063
Digital Object Identifier: doi:10.1214/aop/1039639370
Project Euclid: euclid.aop/1039639370
[11] Hajek, B. (1982). Hitting-time and occupation-time bounds implied by drift analysis with applications. Ann. Appl. Prob. 14 502–525.
Mathematical Reviews (MathSciNet):
MR665291
Zentralblatt MATH:
0495.60094
Digital Object Identifier: doi:10.2307/1426671
JSTOR: links.jstor.org
[12] Has’minskii, R. Z. (1980). Stochastic Stability of Differential Equations. Monographs and Textbooks on Mechanics of Solids and Fluids: Mechanics and Analysis 7. Germantown, Md.
Mathematical Reviews (MathSciNet):
MR600653
[13] Karlin, S. and Taylor, H. (1981). A Second Course in Stochastic Processes. Academic Press, San Diego.
Mathematical Reviews (MathSciNet):
MR611513
Zentralblatt MATH:
0469.60001
[14] Karatzas, I. and Shreve, S. E. (1991). Brownian Motion and Stochastic Calculus. Springer.
Mathematical Reviews (MathSciNet):
MR1121940
[15] Kingman, J. F. C. (1962). Some inequalities for the queue GI/G/1. Biometrica 49 315–324.
[16] Kumar, P. R. and Meyn, S. P. (1996). Duality and linear programs for sta- bility and performance analysis of queueing networks and scheduling policies. IEEE Trans. on Automatic Control 41 4–17.
Mathematical Reviews (MathSciNet):
MR1372612
Digital Object Identifier: doi:10.1109/9.481604
[17] Kumar, S. and Kumar, P. R. (1994). Performance bounds for queueing networks and scheduling policies. IEEE Trans. on Automatic Control 39 1600–1611.
Mathematical Reviews (MathSciNet):
MR1287267
Digital Object Identifier: doi:10.1109/9.310033
[18] Lasserre, J. (2002). Bounds on measures satisfying moment conditions. Ann. Appl. Probab. 12 1114–1137.
Mathematical Reviews (MathSciNet):
MR1925454
Zentralblatt MATH:
1073.90534
Digital Object Identifier: doi:10.1214/aoap/1031863183
Project Euclid: euclid.aoap/1031863183
[19] Lions, P.-L. and Sznitman, A.-S. (1984). Stochastic differential equations with reflecting boundary conditions. Comm. Pure Appl. Math. 37 511–537.
Mathematical Reviews (MathSciNet):
MR745330
Zentralblatt MATH:
0598.60060
Digital Object Identifier: doi:10.1002/cpa.3160370408
[20] Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Communications and Control Engineering Series. Springer-Verlag, London, 1993.
Mathematical Reviews (MathSciNet):
MR1287609
[21] Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markovian processes III: Foster–Lyapunov criteria for continuous time processes. Adv. Appl. Probab. 25 518–548.
[22] Sigman, K. and Yao, D. (1993). Finite moments for inventory processes. Ann. Appl. Probab. 3 765–778.
Mathematical Reviews (MathSciNet):
MR1284984
Zentralblatt MATH:
0817.60094
Digital Object Identifier: doi:10.1214/aoap/1177004970
Project Euclid: euclid.aoap/1177004970
[23] Taylor, L. M. and Williams, R. J. (1993). Existence and uniqueness of semi-martingale reflecting Brownian motion in the orthant. Probab. Theory Related Fields 96 283–317.
Mathematical Reviews (MathSciNet):
MR1231926
Zentralblatt MATH:
0794.60079
Digital Object Identifier: doi:10.1007/BF01292674
[24] Tweedie, R. (1983). The existence of moments for stationary Markov chains. J. Appl. Probab. 20 191–196.
Mathematical Reviews (MathSciNet):
MR688095
Zentralblatt MATH:
0513.60067
Digital Object Identifier: doi:10.2307/3213735
JSTOR: links.jstor.org
Institute of Mathematical Statistics Collections
