Lyapunov drift is a powerful tool for optimizing stochastic queueing networks subject to stability. However, the most convenient drift conditions often provide results in terms of a time average expectation, rather than a pure time average. This paper provides an extended drift-plus-penalty result that ensures stability with desired time averages with probability 1. The analysis uses the law of large numbers for martingale differences. This is applied to quadratic and subquadratic Lyapunov methods for minimizing the time average of a network penalty function subject to stability and to additional time average constraints. Similar to known results for time average expectations, this paper shows that pure time average penalties can be pushed arbitrarily close to optimality, with a corresponding tradeoff in average queue size. Further, in the special case of quadratic Lyapunov functions, the basic drift condition is shown to imply all major forms of queue stability.
References
L. Tassiulas and A. Ephremides, “Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks,” IEEE Transactions on Automatic Control, vol. 37, no. 12, pp. 1936–1948, 1992. MR1200609 10.1109/9.182479 0771.60070 L. Tassiulas and A. Ephremides, “Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks,” IEEE Transactions on Automatic Control, vol. 37, no. 12, pp. 1936–1948, 1992. MR1200609 10.1109/9.182479 0771.60070
L. Tassiulas and A. Ephremides, “Dynamic server allocation to parallel queues with randomly varying connectivity,” IEEE Transactions on Information Theory, vol. 39, no. 2, pp. 466–478, 1993. MR1224342 10.1109/18.212277 0800.94059 L. Tassiulas and A. Ephremides, “Dynamic server allocation to parallel queues with randomly varying connectivity,” IEEE Transactions on Information Theory, vol. 39, no. 2, pp. 466–478, 1993. MR1224342 10.1109/18.212277 0800.94059
M. J. Neely, Dynamic power allocation and routing for satellite and wireless networks with time varying channels [Ph.D. thesis], Massachusetts Institute of Technology, LIDS, Cambridge, Mass, USA, 2003. M. J. Neely, Dynamic power allocation and routing for satellite and wireless networks with time varying channels [Ph.D. thesis], Massachusetts Institute of Technology, LIDS, Cambridge, Mass, USA, 2003.
L. Georgiadis, M. J. Neely, and L. Tassiulas, “Resource allocation and cross-layer control in wireless networks,” Foundations and Trends in Networking, vol. 1, no. 1, pp. 1–144, 2006. L. Georgiadis, M. J. Neely, and L. Tassiulas, “Resource allocation and cross-layer control in wireless networks,” Foundations and Trends in Networking, vol. 1, no. 1, pp. 1–144, 2006.
D. Williams, Probability with Martingales, Cambridge Mathematical Textbooks, Cambridge University Press, Cambridge, UK, 1991. MR1155402 0722.60001 D. Williams, Probability with Martingales, Cambridge Mathematical Textbooks, Cambridge University Press, Cambridge, UK, 1991. MR1155402 0722.60001
S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, Communications and Control Engineering Series, Springer, London, UK, 1993. MR1287609 S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, Communications and Control Engineering Series, Springer, London, UK, 1993. MR1287609
J. G. Dai, “On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models,” The Annals of Applied Probability, vol. 5, no. 1, pp. 49–77, 1995. MR1325041 0822.60083 10.1214/aoap/1177004828 euclid.aoap/1177004828
J. G. Dai, “On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models,” The Annals of Applied Probability, vol. 5, no. 1, pp. 49–77, 1995. MR1325041 0822.60083 10.1214/aoap/1177004828 euclid.aoap/1177004828
M. J. Neely, “Energy optimal control for time-varying wireless networks,” IEEE Transactions on Information Theory, vol. 52, no. 7, pp. 2915–2934, 2006. MR2239991 10.1109/TIT.2006.876219 M. J. Neely, “Energy optimal control for time-varying wireless networks,” IEEE Transactions on Information Theory, vol. 52, no. 7, pp. 2915–2934, 2006. MR2239991 10.1109/TIT.2006.876219
M. J. Neely, “Universal scheduling for networks with arbitrary traffic, channels, and mobility,” in Proceedings of the 49th IEEE Conference on Decision and Control (CDC '10), pp. 1822–1829, Atlanta, Ga, USA, December 2010. M. J. Neely, “Universal scheduling for networks with arbitrary traffic, channels, and mobility,” in Proceedings of the 49th IEEE Conference on Decision and Control (CDC '10), pp. 1822–1829, Atlanta, Ga, USA, December 2010.
R. Agrawal and V. Subramanian, “Optimality of certain channel aware scheduling policies,” in Proceedings of the 40th Annual Allerton Conference on Communication, Control, and Computing, Monticello, Ill, USA, October 2002. R. Agrawal and V. Subramanian, “Optimality of certain channel aware scheduling policies,” in Proceedings of the 40th Annual Allerton Conference on Communication, Control, and Computing, Monticello, Ill, USA, October 2002.
H. Kushner and P. Whiting, “Asymptotic properties of proportional-fair sharing algorithms,” in Proceedings of the 40th Annual Allerton Conference on Communication, Control, and Computing, May 2002. H. Kushner and P. Whiting, “Asymptotic properties of proportional-fair sharing algorithms,” in Proceedings of the 40th Annual Allerton Conference on Communication, Control, and Computing, May 2002.
A. L. Stolyar, “Maximizing queueing network utility subject to stability: greedy primal-dual algorithm,” Queueing Systems, vol. 50, no. 4, pp. 401–457, 2005. MR2172908 10.1007/s11134-005-1450-0 1086.60059 A. L. Stolyar, “Maximizing queueing network utility subject to stability: greedy primal-dual algorithm,” Queueing Systems, vol. 50, no. 4, pp. 401–457, 2005. MR2172908 10.1007/s11134-005-1450-0 1086.60059
Q. Li and R. Negi, “Scheduling in wireless networks under uncertainties: a greedy primal-dual approach,” Tech. Rep., 2010, http://arxiv.org/abs/1001.20502010. http://arxiv.org/abs/1001.20502010 Q. Li and R. Negi, “Scheduling in wireless networks under uncertainties: a greedy primal-dual approach,” Tech. Rep., 2010, http://arxiv.org/abs/1001.20502010. http://arxiv.org/abs/1001.20502010
A. Eryilmaz and R. Srikant, “Fair resource allocation in wireless networks using queue-length-based scheduling and congestion control,” IEEE/ACM Transactions on Networking, vol. 15, no. 6, pp. 1333–1344, 2007. A. Eryilmaz and R. Srikant, “Fair resource allocation in wireless networks using queue-length-based scheduling and congestion control,” IEEE/ACM Transactions on Networking, vol. 15, no. 6, pp. 1333–1344, 2007.
X. Lin and N. B. Shroff, “Joint rate control and scheduling in multihop wireless networks,” in Proceedings of the 43rd IEEE Conference on Decision and Control, pp. 1484–1489, Paradise Island, The Bahamas, December 2004. X. Lin and N. B. Shroff, “Joint rate control and scheduling in multihop wireless networks,” in Proceedings of the 43rd IEEE Conference on Decision and Control, pp. 1484–1489, Paradise Island, The Bahamas, December 2004.
M. Andrews, K. Kumaran, K. Ramanan, A. Stolyar, P. Whiting, and R. Vijayakumar, “Providing quality of service over a shared wireless link,” IEEE Communications Magazine, vol. 39, no. 2, pp. 150–154, 2001. M. Andrews, K. Kumaran, K. Ramanan, A. Stolyar, P. Whiting, and R. Vijayakumar, “Providing quality of service over a shared wireless link,” IEEE Communications Magazine, vol. 39, no. 2, pp. 150–154, 2001.
E. Leonardi, M. Mellia, F. Neri, and M. Ajmone Marsan, “Bounds on average delays and queue size averages and variances in input-queued cell-based switches,” in Proceedings of the 20th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM '01), pp. 1095–1103, Anchorage, Alaska, USA, April 2001. E. Leonardi, M. Mellia, F. Neri, and M. Ajmone Marsan, “Bounds on average delays and queue size averages and variances in input-queued cell-based switches,” in Proceedings of the 20th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM '01), pp. 1095–1103, Anchorage, Alaska, USA, April 2001.
M. J. Neely, E. Modiano, and C. E. Rohrs, “Dynamic power allocation and routing for time-varying wireless networks,” IEEE Journal on Selected Areas in Communications, vol. 23, no. 1, pp. 89–103, 2005. M. J. Neely, E. Modiano, and C. E. Rohrs, “Dynamic power allocation and routing for time-varying wireless networks,” IEEE Journal on Selected Areas in Communications, vol. 23, no. 1, pp. 89–103, 2005.
N. Kahale and P. E. Wright, “Dynamic global packet routing in wireless networks,” in Proceedings of the 16th IEEE Annual Conference on Computer Communications (INFOCOM '97), pp. 1414–1421, April 1997. N. Kahale and P. E. Wright, “Dynamic global packet routing in wireless networks,” in Proceedings of the 16th IEEE Annual Conference on Computer Communications (INFOCOM '97), pp. 1414–1421, April 1997.
S. Shakkottai, R. Srikant, and A. L. Stolyar, “Pathwise optimality of the exponential scheduling rule for wireless channels,” Advances in Applied Probability, vol. 36, no. 4, pp. 1021–1045, 2004. MR2119854 1065.60138 10.1239/aap/1103662957 euclid.aap/1103662957
S. Shakkottai, R. Srikant, and A. L. Stolyar, “Pathwise optimality of the exponential scheduling rule for wireless channels,” Advances in Applied Probability, vol. 36, no. 4, pp. 1021–1045, 2004. MR2119854 1065.60138 10.1239/aap/1103662957 euclid.aap/1103662957
S. P. Meyn, “Stability and asymptotic optimality of generalized maxweight policies,” SIAM Journal on Control and Optimization, vol. 47, no. 6, pp. 3259–3294, 2008/09. MR2476439 1175.90182 10.1137/06067746X S. P. Meyn, “Stability and asymptotic optimality of generalized maxweight policies,” SIAM Journal on Control and Optimization, vol. 47, no. 6, pp. 3259–3294, 2008/09. MR2476439 1175.90182 10.1137/06067746X
A. L. Stolyar, “Maxweight scheduling in a generalized switch: state space collapse and workload minimization in heavy traffic,” The Annals of Applied Probability, vol. 14, no. 1, pp. 1–53, 2004. MR2023015 1057.60092 10.1214/aoap/1075828046 euclid.aoap/1075828046
A. L. Stolyar, “Maxweight scheduling in a generalized switch: state space collapse and workload minimization in heavy traffic,” The Annals of Applied Probability, vol. 14, no. 1, pp. 1–53, 2004. MR2023015 1057.60092 10.1214/aoap/1075828046 euclid.aoap/1075828046
D. Shah and D. Wischik, “Optimal scheduling algorithms for input-queued switches,” in Proceedings of the 25th IEEE International Conference on Computer Communications (INFOCOM '06), pp. 1–11, Barcelona, Spain, April 2006. D. Shah and D. Wischik, “Optimal scheduling algorithms for input-queued switches,” in Proceedings of the 25th IEEE International Conference on Computer Communications (INFOCOM '06), pp. 1–11, Barcelona, Spain, April 2006.
T. Ji, E. Athanasopoulou, and R. Srikant, “Optimal scheduling policies in small generalized switches,” in Proceedings of the 28th IEEE Conference on Computer Communications (INFOCOM '09), pp. 2921–2925, Rio De Janiero, Brazil, April 2009. T. Ji, E. Athanasopoulou, and R. Srikant, “Optimal scheduling policies in small generalized switches,” in Proceedings of the 28th IEEE Conference on Computer Communications (INFOCOM '09), pp. 2921–2925, Rio De Janiero, Brazil, April 2009.
V. J. Venkataramanan and X. Lin, “Structural properties of LDP for queue-length based wireless scheduling algorithms,” in Proceedings of the 45th Annual Allerton Conference on Communication, Control, and Computing, Monticello, Ill, USA, September 2007. V. J. Venkataramanan and X. Lin, “Structural properties of LDP for queue-length based wireless scheduling algorithms,” in Proceedings of the 45th Annual Allerton Conference on Communication, Control, and Computing, Monticello, Ill, USA, September 2007.
V. Lau and C. H. Koh, “Tradeoff analysis of delay-power-CSIT quality of generalized dynamic backpressure algorithm for energy efficient OFDM systems,” in Proceedings of the IEEE International Symposium on Information Theory, pp. 1287–1291, St Petersburg, Russia, August 2011. MR2960493 10.1109/TSP.2012.2198817 V. Lau and C. H. Koh, “Tradeoff analysis of delay-power-CSIT quality of generalized dynamic backpressure algorithm for energy efficient OFDM systems,” in Proceedings of the IEEE International Symposium on Information Theory, pp. 1287–1291, St Petersburg, Russia, August 2011. MR2960493 10.1109/TSP.2012.2198817
P. W. Glynn and S. P. Meyn, “A Liapounov bound for solutions of the Poisson's equation,” The Annals of Probability, vol. 24, no. 2, pp. 916–931, 1996. MR1404536 0863.60063 10.1214/aop/1039639370 euclid.aop/1039639370
P. W. Glynn and S. P. Meyn, “A Liapounov bound for solutions of the Poisson's equation,” The Annals of Probability, vol. 24, no. 2, pp. 916–931, 1996. MR1404536 0863.60063 10.1214/aop/1039639370 euclid.aop/1039639370
A. Shwartz and A. M. Makowski, “Comparing policies in Markov decision processes: Mandl's lemma revisited,” Mathematics of Operations Research, vol. 15, no. 1, pp. 155–174, 1990. MR1038239 0714.90097 10.1287/moor.15.1.155 A. Shwartz and A. M. Makowski, “Comparing policies in Markov decision processes: Mandl's lemma revisited,” Mathematics of Operations Research, vol. 15, no. 1, pp. 155–174, 1990. MR1038239 0714.90097 10.1287/moor.15.1.155
A. M. Makowski and A. Shwartz, “The Poisson equation for countable Markov chains: probabilistic methods and interpretations,” in Handbook of Markov Decision Processes: Methods and Applications, E. A. Feinberg and A. Shwartz, Eds., vol. 40, pp. 269–303, Kluwer Academic, Boston, Mass, USA, 2002. MR1887206 1014.90111 10.1007/978-1-4615-0805-2_9 A. M. Makowski and A. Shwartz, “The Poisson equation for countable Markov chains: probabilistic methods and interpretations,” in Handbook of Markov Decision Processes: Methods and Applications, E. A. Feinberg and A. Shwartz, Eds., vol. 40, pp. 269–303, Kluwer Academic, Boston, Mass, USA, 2002. MR1887206 1014.90111 10.1007/978-1-4615-0805-2_9
Y. S. Chow, “On a strong law of large numbers for martingales,” Annals of Mathematical Statistics, vol. 38, no. 2, article 610, 1967. MR208648 0168.16103 Y. S. Chow, “On a strong law of large numbers for martingales,” Annals of Mathematical Statistics, vol. 38, no. 2, article 610, 1967. MR208648 0168.16103
L. Huang and M. J. Neely, “Delay reduction via Lagrange multipliers in stochastic network optimization,” IEEE Transactions on Automatic Control, vol. 56, no. 4, pp. 842–857, 2011. MR2815394 10.1109/TAC.2010.2067371 L. Huang and M. J. Neely, “Delay reduction via Lagrange multipliers in stochastic network optimization,” IEEE Transactions on Automatic Control, vol. 56, no. 4, pp. 842–857, 2011. MR2815394 10.1109/TAC.2010.2067371
