Statistical Science

Bandwidth Estimation for Best-Effort Internet Traffic

Jin Cao, William S. Cleveland, and Don X. Sun

Full-text: Open access

Abstract

A fundamental problem of Internet traffic engineering is bandwidth estimation: determining the bandwidth (bits per second) required to carry traffic with a specific bit rate (bits per second) offered to an Internet link and satisfy quality-of-service requirements. The traffic is packets of varying sizes that arrive for transmission on the link. Packets can queue up and are dropped if the queue size (bits) is bigger than the size of the buffer (bits) for the queue. For the predominant traffic on the Internet, best-effort traffic, quality metrics are the packet loss (fraction of lost packets), a queueing delay (seconds) and the delay probability (probability of a packet exceeding the delay). This article presents an introduction to bandwidth estimation and a solution to the problem of best-effort traffic for the case where the quality criteria specify negligible packet loss. The solution is a simple statistical model: (1) a formula for the bandwidth as a function of the delay, the delay probability, the traffic bit rate and the mean number of active host-pair connections of the traffic and (2) a random error term. The model is built and validated using queueing theory and extensive empirical study; it is valid for traffic with 64 host-pair connections or more, which is about 1 megabit/s of traffic. The model provides for Internet best-effort traffic what the Erlang delay formula provides for queueing systems with Poisson arrivals and i.i.d. exponential service times.

Article information

Source
Statist. Sci., Volume 19, Number 3 (2004), 518-543.

Dates
First available in Project Euclid: 16 March 2005

Permanent link to this document
https://projecteuclid.org/euclid.ss/1110999313

Digital Object Identifier
doi:10.1214/088342304000000260

Mathematical Reviews number (MathSciNet)
MR2185629

Zentralblatt MATH identifier
1100.62627

Keywords
Queueing Erlang delay formula nonlinear time series long-range dependence QoS statistical multiplexing Internet traffic capacity planning

Citation

Cao, Jin; Cleveland, William S.; Sun, Don X. Bandwidth Estimation for Best-Effort Internet Traffic. Statist. Sci. 19 (2004), no. 3, 518--543. doi:10.1214/088342304000000260. https://projecteuclid.org/euclid.ss/1110999313


Export citation

References

  • Becker, R. A., Cleveland, W. S. and Shyu, M. J. (1996). The visual design and control of trellis display. J. Comput. Graph. Statist. 5 123--155.
  • Ben Fred, S., Bonald, T., Proutiere, A., Régnié, G. and Roberts, J. W. (2001). Statistical bandwidth sharing: A study of congestion at flow level. In Proc. ACM SIGCOMM 2001 111--122. ACM Press, New York.
  • Berger, A. W. and Whitt, W. (1998). Effective bandwidths with priorities. IEEE/ACM Transactions on Networking 6 447--460.
  • Boots, N. and Mandjes, M. (2002). Fast simulation of a queue fed by a superposition of many (heavy-tailed) sources. Probab. Engrg. Inform. Sci. 16 205--232.
  • Botvich, D. D. and Duffield, N. G. (1995). Large deviations, the shape of the loss curve, and economies of scale in large multiplexers. Queueing Systems Theory Appl. 20 293--320.
  • Cáceres, R., Duffield, N., Feldmann, A., Friedmann, J., Greenberg, A., Greer, R., Johnson, T., Kalmanek, C., Krishnamurthy, B., Lavelle, D., Mishra, P., Rexford, J., Ramakrishnan, K., True, F. and van der Merwe, J. (2000). Measurement and analysis of IP network usage and behavior. IEEE Communications Magazine 38(5) 144--151.
  • Cao, J., Cleveland, W. S., Lin, D. and Sun, D. X. (2001). On the nonstationarity of Internet traffic. In Proc. ACM SIGMETRICS 2001 102--112. ACM Press, New York.
  • Cao, J., Cleveland, W. S., Lin, D. and Sun, D. X. (2003). Internet traffic tends toward Poisson and independent as the load increases. Nonlinear Estimation and Classification. Lecture Notes in Statist. 171 83--109. Springer, New York.
  • Cao, J., Cleveland, W. S. and Sun, D. X. (2004). Fractional sum-difference models for open-loop generation of Internet packet traffic. Technical report, Bell Labs, Murray Hill, NJ.
  • Cao, J. and Ramanan, K. (2002). A Poisson limit for buffer overflow probabilities. In Proc. IEEE INFOCOM 2002 2 994--1003. IEEE Press, New York.
  • Chang, C.-S., Chiu, Y.-M. and Song, W. T. (2001). On the performance of multiplexing independent regulated inputs. In Proc. ACM SIGMETRICS 2001 184--193. ACM Press, New York.
  • Chang, C.-S. and Thomas, J. (1995). Effective bandwidth in high-speed digital networks. IEEE J. Selected Areas in Communications 13 1091--1100.
  • Choe, J. and Shroff, N. (1998). A central-limit-theorem-based approach analyzing queue behavior in high-speed networks. IEEE/ACM Transactions on Networking 6 659--671.
  • Choudhury, G. L., Lucantoni, D. M. and Whitt, W. (1994). On the effectiveness of effective bandwidths for admission control in ATM networks. In Proc. 14th Internat. Teletraffic Congress (J. Labetoulle and J. W. Roberts, eds.) 411--420. North-Holland, Amsterdam.
  • Claffy, K., Braun, H.-W. and Polyzos, G. (1995). A parameterizable methodology for Internet traffic flow profiling. IEEE J. Selected Areas in Communications 13 1481--1494.
  • Cooper, R. B. (1972). Introduction to Queueing Theory. Macmillan, New York.
  • Courcoubetis, C. and Siris, V. A. (2001). Procedures and tools for analysis of network traffic measurements. Technical report.
  • Courcoubetis, C., Siris, V. A. and Stamoulis, G. D. (1999). Application of the many sources asymptotic and effective bandwidths to traffic engineering. Telecommunication Systems 12 167--191.
  • Courcoubetis, C. and Weber, R. (1996). Buffer overflow asymptotics for a buffer handling many traffic sources. J. Appl. Probab. 33 886--903.
  • de Veciana, G., Konstantopoulos, T. and Lee, T.-J. (2001). Stability and performance analysis of networks supporting elastic services. IEEE/ACM Transactions on Networking 9 2--14.
  • Duffield, N. G. (1996). Economies of scale in queues with sources having power-law large deviations scalings. J. Appl. Probab. 33 840--857.
  • Elwalid, A., Heyman, D., Lakshman, T. V., Mitra, D. and Weiss, A. (1995). Fundamental bounds and approximations for ATM multiplexers with applications to video teleconferencing. IEEE J. Selected Areas in Communications 13 1004--1016.
  • Elwalid, A. and Mitra, D. (1993). Effective bandwidth of general Markovian traffic sources and admission control of high speed networks. IEEE/ACM Transactions on Networking 1 329--343.
  • Elwalid, A., Mitra, D. and Wentworth, R. H. (1995). A new approach for allocating buffers and bandwidth to heterogeneous regulated traffic in an ATM node. IEEE J. Selected Areas in Communications 13 1115--1127.
  • Erramilli, A., Narayan, O., Neidhardt, A. and Saniee, I. (2000). Performance impacts of multi-scaling in wide area TCP/IP traffic. In Proc. IEEE INFOCOM 2000 1 352--359. IEEE Press, New York.
  • Erramilli, A., Narayan, O. and Willinger, W. (1996). Experimental queueing analysis with long-range dependent packet traffic. IEEE/ACM Transactions on Networking 4 209--223.
  • Fraleigh, C., Tobagi, F. and Diot, C. (2003). Provisioning IP backbone networks to support latency sensitive traffic. In Proc. IEEE INFOCOM 2003 1 375--385. IEEE Press, New York.
  • Gao, J. and Rubin, I. (2001). Multiplicative multifractal modeling of long-range-dependent network traffic. International J. Communication Systems 14 783--801.
  • Gibbens, R. J. and Teh, Y. C. (1999). Critical time and space scales for statistical multiplexing in multiservice networks. In Proc. 16th Internat. Teletraffic Congress (P. Key and D. Smith, eds.) 87--96. North-Holland, Amsterdam.
  • Guerin, R., Ahmadi, H. and Naghshineh, M. (1991). Equivalent capacity and its application to bandwidth allocation in high-speed networks. IEEE J. Selected Areas in Communications 9 968--981.
  • Hosking, J. R. M. (1981). Fractional differencing. Biometrika 68 165--176.
  • Hui, J. Y. (1988). Resource allocation for broadband networks. IEEE J. Selected Areas in Communications 6 1598--1608.
  • Iyer, S., Bhattacharyya, S., Taft, N. and Diot, C. (2003). An approach to alleviate link overload as observed on an IP backbone. In Proc. IEEE INFOCOM 2003 1 406--416. IEEE Press, New York.
  • Kelly, F. (1996). Notes on effective bandwidths. In Stochastic Networks: Theory and Applications (F. P. Kelly, S. Zachary and I. Ziedins, eds.) 141--168. Oxford Univ. Press.
  • Kesidis, G. and Konstantopoulos, T. (2000). Worst-case performance of a buffer with independent shaped arrival processes. IEEE Communications Letters 4 26--28.
  • Kesidis, G., Walrand, J. and Chang, C.-S. (1993). Effective bandwidths for multiclass Markov fluids and other ATM sources. IEEE/ACM Transactions on Networking 1 424--428.
  • Knightly, E. W. (1997). Second moment resource allocation in multi-service networks. In Proc. ACM SIGMETRICS 1997 181--191. ACM Press, New York.
  • Knightly, E. W. and Shroff, N. B. (1999). Admission control for statistical QoS: Theory and practice. IEEE Network 13(2) 20--29.
  • Konstantopoulos, T. and Lin, S.-J. (1996). High variability versus long-range dependence for network performance. In Proc. 35th IEEE Decision and Control 2 1354--1359. IEEE Press, New York.
  • Kontovasilis, K., Wittevrongel, S., Bruneel, H., Van Houdt, B. and Blondia, C. (2002). Performance of telecommunication systems: Selected topics. In Communication Systems: The State of the Art (L. Chapin, ed.). Kluwer, Dordrecht.
  • Leland, W., Taqqu, M., Willinger, W. and Wilson, D. (1994). On the self-similar nature of Ethernet traffic. IEEE/ACM Transactions on Networking 2 1--15.
  • Likhanov, N. and Mazumdar, R. (1998). Cell loss asymptotics in buffers fed with a large number of independent stationary sources. In Proc. IEEE INFOCOM 1998 1 339--346. IEEE Computer Society Press, Los Alamitos, CA.
  • Lo Presti, F., Zhang, Z.-L., Kurose, J. and Towsley, D. (1999). Source time scale and optimal buffer/bandwidth tradeoff for heterogeneous regulated traffic in a network node. IEEE/ACM Transactions on Networking 7 490--501.
  • Mandjes, M. and Boots, N. (2002). The shape of the loss curve, and the impact of long-range dependence on network performance. Technical report.
  • Mandjes, M. and Boots, N. (2004). The shape of the loss curve, and the impact of long-range dependence on network performance. AEÜ International J. Electronics and Communications 58 101--117.
  • Mandjes, M. and Kim, J. H. (2001). Large deviations for small buffers: An insensitivity result. Queueing Syst. 37 349--362.
  • Mosteller, F. and Tukey, J. W. (1977). Data Analysis and Regression: A Second Course in Statistics. Addison--Wesley, Reading, MA.
  • Norros, I. (1994). A storage model with self-similar input. Queueing Systems Theory Appl. 16 387--396.
  • Paxson, V. (1997). Automated packet trace analysis of TCP implementations. In Proc. ACM SIGCOMM 1997 167--179. ACM Press, New York.
  • Paxson, V. and Floyd, S. (1995). Wide-area traffic: The failure of Poisson modeling. IEEE/ACM Transactions on Networking 3 226--244.
  • Ribeiro, V. J., Riedi, R. H., Crouse, M. S. and Baraniuk, R. G. (1999). Simulation of non-Gaussian long-range-dependent traffic using wavelets. In Proc. ACM SIGMETRICS 1999 1--12. ACM Press, New York.
  • Riedi, R. H., Crouse, M. S., Ribeiro, V. J. and Baraniuk, R. G. (1999). A multifractal wavelet model with application to network traffic. IEEE Trans. Inform. Theory 45 992--1019.
  • Shroff, N. B. and Schwartz, M. (1998). Improved loss calculations at an ATM multiplexer. IEEE/ACM Transactions on Networking 6 411--421.
  • Simonian, A. and Guibert, J. (1995). Large deviations approximation for fluid queues fed by a large number of on/off sources. IEEE J. Selected Areas in Communications 13 1017--1027.
  • Stevens, W. R. (1994). TCP/IP Illustrated 1. Addison--Wesley, Reading, MA.
  • Zhang, Z.-L., Towsley, D. and Kurose, J. (1994). Statistical analysis of generalized processor sharing scheduling discipline. In Proc. ACM SIGCOMM 1994 68--77. ACM Press, New York.