Bandwidth Estimation for Best-Effort Internet Traffic
Jin Cao, William S. Cleveland, and Don X. Sun
Source: Statist. Sci. Volume 19, Number 3
(2004), 518-543.
Abstract
A fundamental problem of Internet traffic engineering is bandwidth estimation: determining the bandwidth (bits per second) required to carry traffic with a specific bit rate (bits per second) offered to an Internet link and satisfy quality-of-service requirements. The traffic is packets of varying sizes that arrive for transmission on the link. Packets can queue up and are dropped if the queue size (bits) is bigger than the size of the buffer (bits) for the queue. For the predominant traffic on the Internet, best-effort traffic, quality metrics are the packet loss (fraction of lost packets), a queueing delay (seconds) and the delay probability (probability of a packet exceeding the delay). This article presents an introduction to bandwidth estimation and a solution to the problem of best-effort traffic for the case where the quality criteria specify negligible packet loss. The solution is a simple statistical model: (1) a formula for the bandwidth as a function of the delay, the delay probability, the traffic bit rate and the mean number of active host-pair connections of the traffic and (2) a random error term. The model is built and validated using queueing theory and extensive empirical study; it is valid for traffic with 64 host-pair connections or more, which is about 1 megabit/s of traffic. The model provides for Internet best-effort traffic what the Erlang delay formula provides for queueing systems with Poisson arrivals and i.i.d. exponential service times.
First Page:
Show
Hide
Keywords: Queueing; Erlang delay formula; nonlinear time series; long-range dependence; QoS; statistical multiplexing; Internet traffic; capacity planning
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.ss/1110999313
Digital Object Identifier: doi:10.1214/088342304000000260
Mathematical Reviews number (MathSciNet): MR2185629
Zentralblatt MATH identifier: 1100.62627
References
Becker, R. A., Cleveland, W. S. and Shyu, M. J. (1996). The visual design and control of trellis display. J. Comput. Graph. Statist. 5 123--155.
Ben Fred, S., Bonald, T., Proutiere, A., Régnié, G. and Roberts, J. W. (2001). Statistical bandwidth sharing: A study of congestion at flow level. In Proc. ACM SIGCOMM 2001 111--122. ACM Press, New York.
Berger, A. W. and Whitt, W. (1998). Effective bandwidths with priorities. IEEE/ACM Transactions on Networking 6 447--460.
Boots, N. and Mandjes, M. (2002). Fast simulation of a queue fed by a superposition of many (heavy-tailed) sources. Probab. Engrg. Inform. Sci. 16 205--232.
Mathematical Reviews (MathSciNet): MR1891473
Digital Object Identifier: doi:10.1017/S026996480216205X
Botvich, D. D. and Duffield, N. G. (1995). Large deviations, the shape of the loss curve, and economies of scale in large multiplexers. Queueing Systems Theory Appl. 20 293--320.
Mathematical Reviews (MathSciNet): MR1356879
Digital Object Identifier: doi:10.1007/BF01245322
Zentralblatt MATH: 0847.90052
Cáceres, R., Duffield, N., Feldmann, A., Friedmann, J., Greenberg, A., Greer, R., Johnson, T., Kalmanek, C., Krishnamurthy, B., Lavelle, D., Mishra, P., Rexford, J., Ramakrishnan, K., True, F. and van der Merwe, J. (2000). Measurement and analysis of IP network usage and behavior. IEEE Communications Magazine 38(5) 144--151.
Cao, J., Cleveland, W. S., Lin, D. and Sun, D. X. (2001). On the nonstationarity of Internet traffic. In Proc. ACM SIGMETRICS 2001 102--112. ACM Press, New York.
Cao, J., Cleveland, W. S., Lin, D. and Sun, D. X. (2003). Internet traffic tends toward Poisson and independent as the load increases. Nonlinear Estimation and Classification. Lecture Notes in Statist. 171 83--109. Springer, New York.
Mathematical Reviews (MathSciNet): MR2005785
Cao, J., Cleveland, W. S. and Sun, D. X. (2004). Fractional sum-difference models for open-loop generation of Internet packet traffic. Technical report, Bell Labs, Murray Hill, NJ.
Cao, J. and Ramanan, K. (2002). A Poisson limit for buffer overflow probabilities. In Proc. IEEE INFOCOM 2002 2 994--1003. IEEE Press, New York.
Chang, C.-S., Chiu, Y.-M. and Song, W. T. (2001). On the performance of multiplexing independent regulated inputs. In Proc. ACM SIGMETRICS 2001 184--193. ACM Press, New York.
Chang, C.-S. and Thomas, J. (1995). Effective bandwidth in high-speed digital networks. IEEE J. Selected Areas in Communications 13 1091--1100.
Choe, J. and Shroff, N. (1998). A central-limit-theorem-based approach analyzing queue behavior in high-speed networks. IEEE/ACM Transactions on Networking 6 659--671.
Choudhury, G. L., Lucantoni, D. M. and Whitt, W. (1994). On the effectiveness of effective bandwidths for admission control in ATM networks. In Proc. 14th Internat. Teletraffic Congress (J. Labetoulle and J. W. Roberts, eds.) 411--420. North-Holland, Amsterdam.
Claffy, K., Braun, H.-W. and Polyzos, G. (1995). A parameterizable methodology for Internet traffic flow profiling. IEEE J. Selected Areas in Communications 13 1481--1494.
Cooper, R. B. (1972). Introduction to Queueing Theory. Macmillan, New York.
Zentralblatt MATH: 0284.60085
Courcoubetis, C. and Siris, V. A. (2001). Procedures and tools for analysis of network traffic measurements. Technical report.
Courcoubetis, C., Siris, V. A. and Stamoulis, G. D. (1999). Application of the many sources asymptotic and effective bandwidths to traffic engineering. Telecommunication Systems 12 167--191.
Courcoubetis, C. and Weber, R. (1996). Buffer overflow asymptotics for a buffer handling many traffic sources. J. Appl. Probab. 33 886--903.
Mathematical Reviews (MathSciNet): MR1401482
Digital Object Identifier: doi:10.2307/3215366
JSTOR: links.jstor.org
Zentralblatt MATH: 0866.60083
de Veciana, G., Konstantopoulos, T. and Lee, T.-J. (2001). Stability and performance analysis of networks supporting elastic services. IEEE/ACM Transactions on Networking 9 2--14.
Duffield, N. G. (1996). Economies of scale in queues with sources having power-law large deviations scalings. J. Appl. Probab. 33 840--857.
Mathematical Reviews (MathSciNet): MR1401479
Digital Object Identifier: doi:10.2307/3215363
JSTOR: links.jstor.org
Zentralblatt MATH: 0866.60078
Elwalid, A., Heyman, D., Lakshman, T. V., Mitra, D. and Weiss, A. (1995). Fundamental bounds and approximations for ATM multiplexers with applications to video teleconferencing. IEEE J. Selected Areas in Communications 13 1004--1016.
Elwalid, A. and Mitra, D. (1993). Effective bandwidth of general Markovian traffic sources and admission control of high speed networks. IEEE/ACM Transactions on Networking 1 329--343.
Elwalid, A., Mitra, D. and Wentworth, R. H. (1995). A new approach for allocating buffers and bandwidth to heterogeneous regulated traffic in an ATM node. IEEE J. Selected Areas in Communications 13 1115--1127.
Erramilli, A., Narayan, O., Neidhardt, A. and Saniee, I. (2000). Performance impacts of multi-scaling in wide area TCP/IP traffic. In Proc. IEEE INFOCOM 2000 1 352--359. IEEE Press, New York.
Erramilli, A., Narayan, O. and Willinger, W. (1996). Experimental queueing analysis with long-range dependent packet traffic. IEEE/ACM Transactions on Networking 4 209--223.
Fraleigh, C., Tobagi, F. and Diot, C. (2003). Provisioning IP backbone networks to support latency sensitive traffic. In Proc. IEEE INFOCOM 2003 1 375--385. IEEE Press, New York.
Gao, J. and Rubin, I. (2001). Multiplicative multifractal modeling of long-range-dependent network traffic. International J. Communication Systems 14 783--801.
Gibbens, R. J. and Teh, Y. C. (1999). Critical time and space scales for statistical multiplexing in multiservice networks. In Proc. 16th Internat. Teletraffic Congress (P. Key and D. Smith, eds.) 87--96. North-Holland, Amsterdam.
Guerin, R., Ahmadi, H. and Naghshineh, M. (1991). Equivalent capacity and its application to bandwidth allocation in high-speed networks. IEEE J. Selected Areas in Communications 9 968--981.
Hosking, J. R. M. (1981). Fractional differencing. Biometrika 68 165--176.
Mathematical Reviews (MathSciNet): MR614953
Zentralblatt MATH: 0464.62088
Digital Object Identifier: doi:10.2307/2335817
JSTOR: links.jstor.org
Hui, J. Y. (1988). Resource allocation for broadband networks. IEEE J. Selected Areas in Communications 6 1598--1608.
Iyer, S., Bhattacharyya, S., Taft, N. and Diot, C. (2003). An approach to alleviate link overload as observed on an IP backbone. In Proc. IEEE INFOCOM 2003 1 406--416. IEEE Press, New York.
Kelly, F. (1996). Notes on effective bandwidths. In Stochastic Networks: Theory and Applications (F. P. Kelly, S. Zachary and I. Ziedins, eds.) 141--168. Oxford Univ. Press.
Zentralblatt MATH: 0845.00046
Kesidis, G. and Konstantopoulos, T. (2000). Worst-case performance of a buffer with independent shaped arrival processes. IEEE Communications Letters 4 26--28.
Kesidis, G., Walrand, J. and Chang, C.-S. (1993). Effective bandwidths for multiclass Markov fluids and other ATM sources. IEEE/ACM Transactions on Networking 1 424--428.
Knightly, E. W. (1997). Second moment resource allocation in multi-service networks. In Proc. ACM SIGMETRICS 1997 181--191. ACM Press, New York.
Knightly, E. W. and Shroff, N. B. (1999). Admission control for statistical QoS: Theory and practice. IEEE Network 13(2) 20--29.
Konstantopoulos, T. and Lin, S.-J. (1996). High variability versus long-range dependence for network performance. In Proc. 35th IEEE Decision and Control 2 1354--1359. IEEE Press, New York.
Kontovasilis, K., Wittevrongel, S., Bruneel, H., Van Houdt, B. and Blondia, C. (2002). Performance of telecommunication systems: Selected topics. In Communication Systems: The State of the Art (L. Chapin, ed.). Kluwer, Dordrecht.
Leland, W., Taqqu, M., Willinger, W. and Wilson, D. (1994). On the self-similar nature of Ethernet traffic. IEEE/ACM Transactions on Networking 2 1--15.
Likhanov, N. and Mazumdar, R. (1998). Cell loss asymptotics in buffers fed with a large number of independent stationary sources. In Proc. IEEE INFOCOM 1998 1 339--346. IEEE Computer Society Press, Los Alamitos, CA.
Mathematical Reviews (MathSciNet): MR1699600
Digital Object Identifier: doi:10.1239/jap/1032374231
Project Euclid: euclid.jap/1032374231
Zentralblatt MATH: 0944.60097
Lo Presti, F., Zhang, Z.-L., Kurose, J. and Towsley, D. (1999). Source time scale and optimal buffer/bandwidth tradeoff for heterogeneous regulated traffic in a network node. IEEE/ACM Transactions on Networking 7 490--501.
Mandjes, M. and Boots, N. (2002). The shape of the loss curve, and the impact of long-range dependence on network performance. Technical report.
Mandjes, M. and Boots, N. (2004). The shape of the loss curve, and the impact of long-range dependence on network performance. AEÜ International J. Electronics and Communications 58 101--117.
Mandjes, M. and Kim, J. H. (2001). Large deviations for small buffers: An insensitivity result. Queueing Syst. 37 349--362.
Mathematical Reviews (MathSciNet): MR1834125
Digital Object Identifier: doi:10.1023/A:1010837416603
Zentralblatt MATH: 1017.90023
Mosteller, F. and Tukey, J. W. (1977). Data Analysis and Regression: A Second Course in Statistics. Addison--Wesley, Reading, MA.
Norros, I. (1994). A storage model with self-similar input. Queueing Systems Theory Appl. 16 387--396.
Mathematical Reviews (MathSciNet): MR1278465
Digital Object Identifier: doi:10.1007/BF01158964
Zentralblatt MATH: 0811.68059
Paxson, V. (1997). Automated packet trace analysis of TCP implementations. In Proc. ACM SIGCOMM 1997 167--179. ACM Press, New York.
Paxson, V. and Floyd, S. (1995). Wide-area traffic: The failure of Poisson modeling. IEEE/ACM Transactions on Networking 3 226--244.
Ribeiro, V. J., Riedi, R. H., Crouse, M. S. and Baraniuk, R. G. (1999). Simulation of non-Gaussian long-range-dependent traffic using wavelets. In Proc. ACM SIGMETRICS 1999 1--12. ACM Press, New York.
Riedi, R. H., Crouse, M. S., Ribeiro, V. J. and Baraniuk, R. G. (1999). A multifractal wavelet model with application to network traffic. IEEE Trans. Inform. Theory 45 992--1019.
Mathematical Reviews (MathSciNet): MR1682524
Digital Object Identifier: doi:10.1109/18.761337
Zentralblatt MATH: 0947.94003
Shroff, N. B. and Schwartz, M. (1998). Improved loss calculations at an ATM multiplexer. IEEE/ACM Transactions on Networking 6 411--421.
Simonian, A. and Guibert, J. (1995). Large deviations approximation for fluid queues fed by a large number of on/off sources. IEEE J. Selected Areas in Communications 13 1017--1027.
Stevens, W. R. (1994). TCP/IP Illustrated 1. Addison--Wesley, Reading, MA.
Zhang, Z.-L., Towsley, D. and Kurose, J. (1994). Statistical analysis of generalized processor sharing scheduling discipline. In Proc. ACM SIGCOMM 1994 68--77. ACM Press, New York.
