Network Tomography: Recent Developments
Rui Castro, Mark Coates, Gang Liang, Robert Nowak, and Bin Yu
Source: Statist. Sci. Volume 19, Number 3
(2004), 499-517.
Abstract
Today’s Internet is a massive, distributed network which continues to explode in size as e-commerce and related activities grow. The heterogeneous and largely unregulated structure of the Internet renders tasks such as dynamic routing, optimized service provision, service level verification and detection of anomalous/malicious behavior extremely challenging. The problem is compounded by the fact that one cannot rely on the cooperation of individual servers and routers to aid in the collection of network traffic measurements vital for these tasks. In many ways, network monitoring and inference problems bear a strong resemblance to other “inverse problems” in which key aspects of a system are not directly observable. Familiar signal processing or statistical problems such as tomographic image reconstruction and phylogenetic tree identification have interesting connections to those arising in networking. This article introduces network tomography, a new field which we believe will benefit greatly from the wealth of statistical theory and algorithms. It focuses especially on recent developments in the field including the application of pseudo-likelihood methods and tree estimation formulations.
First Page:
Show
Hide
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.ss/1110999312
Digital Object Identifier: doi:10.1214/088342304000000422
Mathematical Reviews number (MathSciNet): MR2185628
Zentralblatt MATH identifier: 1100.62628
References
Berger, J. O., Liseo, B. and Wolpert, R. L. (1999). Integrated likelihood methods for eliminating nuisance parameters (with discussion). Statist. Sci. 14 1--28.
Mathematical Reviews (MathSciNet): MR1702200
Digital Object Identifier: doi:10.1214/ss/1009212518
Project Euclid: euclid.ss/1009211804
Zentralblatt MATH: 1059.62521
Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems (with discussion). J. Roy. Statist. Soc. Ser. B 36 192--236.
Mathematical Reviews (MathSciNet): MR373208
JSTOR: links.jstor.org
Besag, J. (1975). Statistical analysis of non-lattice data. The Statistician 24 179--195.
Bestavros, A., Byers, J. and Harfoush, K. (2002). Inference and labeling of metric-induced network topologies. In Proc. IEEE INFOCOM 2002 2 628--637. IEEE Press, New York.
Blackwell, D. (1973). Approximate normality of large products. Technical report, Dept. Statistics, Univ. California, Berkeley.
Cáceres, R., Duffield, N., Horowitz, J. and Towsley, D. (1999). Multicast-based inference of network-internal loss characteristics. IEEE Trans. Inform. Theory 45 2462--2480.
Mathematical Reviews (MathSciNet): MR1725131
Digital Object Identifier: doi:10.1109/18.796384
Zentralblatt MATH: 0961.94002
Cao, J., Davis, D.,Vander Wiel, S. and Yu, B. (2000a). Time-varying network tomography: Router link data. J. Amer. Statist. Assoc. 95 1063--1075.
Mathematical Reviews (MathSciNet): MR1821715
Digital Object Identifier: doi:10.2307/2669743
JSTOR: links.jstor.org
Zentralblatt MATH: 1072.90507
Cao, J., Vander Wiel, S., Yu, B. and Zhu, Z. (2000b). A scalable method for estimating network traffic matrices. Technical report, Bell Labs.
Castro, R., Coates, M. and Nowak, R. (2004). Likelihood based hierarchical clustering. IEEE Trans. Signal Process. 52 2308--2321.
Mathematical Reviews (MathSciNet): MR2085590
Digital Object Identifier: doi:10.1109/TSP.2004.831124
Chao, X., Miyazawa, M. and Pinedo, M. (1999). Queueing Networks: Customers, Signals and Product Form Solutions. Wiley, New York.
Coates, M., Castro, R., Nowak, R., Gadhiok, M., King, R. and Tsang, Y. (2002a). Maximum likelihood network topology identification from edge-based unicast measurements. In Proc. ACM SIGMETRICS 2002 11--20. ACM Press, New York.
Coates, M., Hero, A., Nowak, R. and Yu, B. (2002b). Internet tomography. IEEE Signal Processing Magazine 19(3) 47--65.
Coates, M. and Nowak, R. (2000). Network loss inference using unicast end-to-end measurement. In Proc. ITC Seminar on IP Traffic, Measurement and Modelling 28-1--28-9. Available at citeseer.ist.psu.edu/context/1699850/514748.
Coates, M. and Nowak, R. (2002). Sequential Monte Carlo inference of internal delays in nonstationary communication networks. IEEE Trans. Signal Process. 50 366--376.
Cox, D. R. (1975). Partial likelihood. Biometrika 62 269--276.
Mathematical Reviews (MathSciNet): MR400509
Zentralblatt MATH: 0312.62002
Digital Object Identifier: doi:10.2307/2335362
JSTOR: links.jstor.org
Csiszár, I. (1975). $I$-divergence geometry of probability distributions and minimization problems. Ann. Probab. 3 146--158.
Mathematical Reviews (MathSciNet): MR365798
Digital Object Identifier: doi:10.1214/aop/1176996454
Zentralblatt MATH: 0318.60013
JSTOR: links.jstor.org
Duffield, N., Horowitz, J. and Lo Presti, F. (2001). Adaptive multicast topology inference. In Proc. IEEE INFOCOM 2001 3 1636--1645. IEEE Press, New York.
Duffield, N., Horowitz, J., Lo Presti, F. and Towsley, D. (2002). Multicast topology inference from measured end-to-end loss. IEEE Trans. Inform. Theory 48 26--45.
Mathematical Reviews (MathSciNet): MR1872164
Digital Object Identifier: doi:10.1109/18.971737
Zentralblatt MATH: 1059.94050
Duffield, N., Lo Presti, F., Paxson, V. and Towsley, D. (2001). Inferring link loss using striped unicast probes. In Proc. IEEE INFOCOM 2001 2 915--923. IEEE Press, New York.
Fasulo, D. (1999). An analysis of recent work on clustering algorithms. Technical Report 01-03-02, Dept. Computer Science and Engineering, Univ. Washington. Available at citeseer. nj.nec.com/fasulo99analysi.html.
Harfoush, K., Bestavros, A. and Byers, J. (2000). Robust identification of shared losses using end-to-end unicast probes. In Proc. IEEE International Conference on Network Protocols 22--33. IEEE Press, New York. Errata available as Technical Report 2001-001, Dept. Computer Science, Boston Univ.
Hastings, W. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57 97--109.
Ihaka, R. and Gentleman, R. (1996). R: A language for data analysis and graphics. J. Comput. Graph. Statist. 5 299--314.
Kelly, F. P., Zachary, S. and Ziedins, I., eds. (1996). Stochastic Networks: Theory and Applications. Oxford Univ. Press.
Zentralblatt MATH: 0845.00046
Leland, W., Taqqu, M., Willinger, W. and Wilson, D. (1994). On the self-similar nature of Ethernet traffic. IEEE/ACM Transactions on Networking 2 1--15.
Liang, G. and Yu, B. (2003a). Maximum pseudo likelihood estimation in network tomography. IEEE Trans. Signal Process. 51 2043--2053.
Liang, G. and Yu, B. (2003b). Pseudo likelihood estimation in network tomography. In Proc. IEEE INFOCOM 2003 3 2101--2111. IEEE Press, New York.
Lo Presti, F., Duffield, N., Horowitz, J. and Towsley, D. (2002). Multicast-based inference of network-internal delay distributions. IEEE/ACM Transactions on Networking 10 761--775.
Morris, R. and Lin, D. (2000). Variance of aggregated web traffic. In Proc. IEEE INFOCOM 2000 1 360--366. IEEE Press, New York.
O'Sullivan, F. (1986). A statistical perspective on ill-posed inverse problems (with discussion). Statist. Sci. 1 502--527.
Mathematical Reviews (MathSciNet): MR874480
JSTOR: links.jstor.org
Digital Object Identifier: doi:10.1214/ss/1177013525
Project Euclid: euclid.ss/1177013525
Zentralblatt MATH: 0625.62110
Padmanabhan, V. N., Qiu, L. and Wang, H. (2002). Passive network tomography using Bayesian inference. In Proc. ACM SIGCOMM Workshop on Internet Measurement 93--94. ACM Press, New York.
Pásztor, A. and Veitch, D. (2002). PC based precision timing without GPS. In Proc. ACM SIGMETRICS 2002 1--10. ACM Press, New York.
Ratnasamy, S. and McCanne, S. (1999). Inference of multicast routing trees and bottleneck bandwidths using end-to-end measurements. In Proc. IEEE INFOCOM 1999 1 353--360. IEEE Press, New York.
Rissanen, J. (1989). Stochastic Complexity in Statistical Inquiry. World Scientific, Singapore.
Mathematical Reviews (MathSciNet): MR1082556
Zentralblatt MATH: 0800.68508
Robert, C. and Casella, G. (1999). Monte Carlo Statistical Methods. Springer, New York.
Mathematical Reviews (MathSciNet): MR1707311
Zentralblatt MATH: 0935.62005
Rolls, D. (2003). Limit theorems and estimation for structural and aggregate teletramc models. Ph.D. dissertation, Queen's Univ., Kingston, Ontario, Canada.
Scott, D. (1992). Multivariate Density Estimation: Theory, Practice and Visualization. Wiley, New York.
Mathematical Reviews (MathSciNet): MR1191168
Zentralblatt MATH: 0850.62006
Shih, M. and Hero, A. (2001). Unicast inference of network link delay distributions from edge measurements. In Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing 6 3421--3424. IEEE Press, New York.
Tebaldi, C. and West, M. (1998). Bayesian inference on network traffic using link count data (with discussion). J. Amer. Statist. Assoc. 93 557--576.
Mathematical Reviews (MathSciNet): MR1631325
Digital Object Identifier: doi:10.2307/2670105
JSTOR: links.jstor.org
Zentralblatt MATH: 1072.62650
Tsang, Y., Coates, M. and Nowak, R. (2001). Passive network tomography using EM algorithms. In Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing 3 1469--1472. IEEE Press, New York.
Tsang, Y., Coates, M. J. and Nowak, R. (2003). Network delay tomography. IEEE Trans. Signal Process. 51 2125--2136.
Vanderbei, R. J. and Iannone, J. (1994). An EM approach to OD matrix estimation. Technical Report SOR 94-04, Princeton Univ.
Vardi, Y. (1996). Network tomography: Estimating source-destination traffic intensities from link data. J. Amer. Statist. Assoc. 91 365--377.
Mathematical Reviews (MathSciNet): MR1394093
Digital Object Identifier: doi:10.2307/2291416
JSTOR: links.jstor.org
Zentralblatt MATH: 0871.62103
Ward, J. H. (1963). Hierarchical grouping to optimize an objective function. J. Amer. Statist. Assoc. 58 236--245.
Mathematical Reviews (MathSciNet): MR148188
Digital Object Identifier: doi:10.2307/2282967
JSTOR: links.jstor.org
White, H. (1994). Estimation, Inference and Specification Analysis. Cambridge Univ. Press, New York.
Mathematical Reviews (MathSciNet): MR1292251
Zentralblatt MATH: 0860.62100
Willet, P. (1988). Recent trends in hierarchical document clustering: A critical review. Information Processing and Management 24 577--597.
Ziotopolous, A., Hero, A. and Wasserman, K. (2001). Estimation of network link loss rates via chaining in multicast trees. In Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing 4 2517--2520. IEEE Press, New York.
