The Annals of Statistics

Pseudo-likelihood methods for community detection in large sparse networks

Arash A. Amini, Aiyou Chen, Peter J. Bickel, and Elizaveta Levina

Full-text: Open access

Abstract

Many algorithms have been proposed for fitting network models with communities, but most of them do not scale well to large networks, and often fail on sparse networks. Here we propose a new fast pseudo-likelihood method for fitting the stochastic block model for networks, as well as a variant that allows for an arbitrary degree distribution by conditioning on degrees. We show that the algorithms perform well under a range of settings, including on very sparse networks, and illustrate on the example of a network of political blogs. We also propose spectral clustering with perturbations, a method of independent interest, which works well on sparse networks where regular spectral clustering fails, and use it to provide an initial value for pseudo-likelihood. We prove that pseudo-likelihood provides consistent estimates of the communities under a mild condition on the starting value, for the case of a block model with two communities.

Article information

Source
Ann. Statist., Volume 41, Number 4 (2013), 2097-2122.

Dates
First available in Project Euclid: 23 October 2013

Permanent link to this document
https://projecteuclid.org/euclid.aos/1382547514

Digital Object Identifier
doi:10.1214/13-AOS1138

Mathematical Reviews number (MathSciNet)
MR3127859

Zentralblatt MATH identifier
1277.62166

Subjects
Primary: 62G20: Asymptotic properties
Secondary: 62H99: None of the above, but in this section

Keywords
Community detection network pseudo-likelihood

Citation

Amini, Arash A.; Chen, Aiyou; Bickel, Peter J.; Levina, Elizaveta. Pseudo-likelihood methods for community detection in large sparse networks. Ann. Statist. 41 (2013), no. 4, 2097--2122. doi:10.1214/13-AOS1138. https://projecteuclid.org/euclid.aos/1382547514


Export citation

References

  • [1] Adamic, L. A. and Glance, N. (2005). The political blogosphere and the 2004 US election. In Proceedings of the WWW-2005 Workshop on the Weblogging Ecosystem. ACM, New York.
  • [2] Airoldi, E. M., Blei, D. M., Fienberg, S. E. and Xing, E. P. (2008). Mixed membership stochastic blockmodels. J. Mach. Learn. Res. 9 1981–2014.
  • [3] Amini, A. A., Chen, A., Bickel, P. J. and Levina, E. (2013). Supplement to “Pseudo-likelihood methods for community detection in large sparse networks.” DOI:10.1214/13-AOS1138SUPP.
  • [4] Ball, B., Karrer, B. and Newman, M. E. J. (2011). An efficient and principled method for detecting communities in networks. Phys. Rev. E (3) 34 036103.
  • [5] Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems. J. R. Stat. Soc. Ser. B Stat. Methodol. 36 192–236.
  • [6] Bickel, P. J. and Chen, A. (2009). A nonparametric view of network models and Newman–Girvan and other modularities. Proc. Natl. Acad. Sci. USA 106 21068–21073.
  • [7] Bickel, P. J., Chen, A. and Levina, E. (2011). The method of moments and degree distributions for network models. Ann. Statist. 39 2280–2301.
  • [8] Bickel, P. J., Choi, D., Chang, X. and Zhang, H. (2012). Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels. Available at arXiv:1207.0865.
  • [9] Bickel, P. J. and Doksum, K. A. (2007). Mathematical Statistics: Basic Ideas and Selected Topics, 2nd ed. Prentice Hall, New York.
  • [10] Celisse, A., Daudin, J.-J. and Pierre, L. (2012). Consistency of maximum-likelihood and variational estimators in the stochastic block model. Electron. J. Stat. 6 1847–1899.
  • [11] Channarond, A., Daudin, J. J. and Robin, S. (2011). Classification and estimation in the stochastic block model based on the empirical degrees. Available at arXiv:1110.6517.
  • [12] Chatterjee, S. (2012). Matrix estimation by universal singular value thresholding. Available at arXiv:1212.1247.
  • [13] Chaudhuri, K., Chung, F. and Tsiatas, A. (2012). Spectral clustering of graphs with general degrees in the extended planted partition model. JMLR Workshop and Conference Proceedings 23 35.1–35.23.
  • [14] Decelle, A., Krzakala, F., Moore, C. and Zdeborová, L. (2012). Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications. Phys. Rev. E (3) 84 066106.
  • [15] Fortunato, S. (2010). Community detection in graphs. Phys. Rep. 486 75–174.
  • [16] Gleser, L. J. (1975). On the distribution of the number of successes in independent trials. Ann. Probab. 3 182–188.
  • [17] Handcock, M. S., Raftery, A. E. and Tantrum, J. M. (2007). Model-based clustering for social networks. J. Roy. Statist. Soc. Ser. A 170 301–354.
  • [18] Hoeffding, W. (1956). On the distribution of the number of successes in independent trials. Ann. Math. Statist. 27 713–721.
  • [19] Hoff, P. D. (2007). Modeling homophily and stochastic equivalence in symmetric relational data. In Advances in Neural Information Processing Systems, Vol. 19. MIT Press, Cambridge, MA.
  • [20] Holland, P. W., Laskey, K. B. and Leinhardt, S. (1983). Stochastic blockmodels: First steps. Social Networks 5 109–137.
  • [21] Holland, P. W. and Leinhardt, S. (1981). An exponential family of probability distributions for directed graphs. J. Amer. Statist. Assoc. 76 33–65.
  • [22] Karrer, B. and Newman, M. E. J. (2011). Stochastic blockmodels and community structure in networks. Phys. Rev. E (3) 83 016107, 10.
  • [23] Mariadassou, M., Robin, S. and Vacher, C. (2010). Uncovering latent structure in valued graphs: A variational approach. Ann. Appl. Stat. 4 715–742.
  • [24] Mossel, E., Neeman, J. and Sly, A. (2012). Stochastic block models and reconstruction. Available at arXiv:1202.1499.
  • [25] Newman, M. E. J. (2004). Detecting community structure in networks. Eur. Phys. J. B 38 321–330.
  • [26] Newman, M. E. J. (2006). Finding community structure in networks using the eigenvectors of matrices. Phys. Rev. E (3) 74 036104, 19.
  • [27] Newman, M. E. J. (2006). Modularity and community structure in networks. Proc. Natl. Acad. Sci. USA 103 8577–8582.
  • [28] Newman, M. E. J. and Girvan, M. (2004). Finding and evaluating community structure in networks. Phys. Rev. E (3) 69 026113.
  • [29] Newman, M. E. J. and Leicht, E. A. (2007). Mixture models and exploratory analysis in networks. Proc. Natl. Acad. Sci. USA 104 9564–9569.
  • [30] Nowicki, K. and Snijders, T. A. B. (2001). Estimation and prediction for stochastic blockstructures. J. Amer. Statist. Assoc. 96 1077–1087.
  • [31] Perry, P. O. and Wolfe, P. J. (2012). Null models for network data. Available at arXiv:1201.5871v1.
  • [32] Rohe, K., Chatterjee, S. and Yu, B. (2011). Spectral clustering and the high-dimensional stochastic blockmodel. Ann. Statist. 39 1878–1915.
  • [33] Shi, J. and Malik, J. (2000). Normalized cuts and image segmentation. IEEE Trans. Pattern Analysis and Machine Intelligence 22 888–905.
  • [34] Snijders, T. A. B. and Nowicki, K. (1997). Estimation and prediction for stochastic blockmodels for graphs with latent block structure. J. Classification 14 75–100.
  • [35] Wang, Y. J. and Wong, G. Y. (1987). Stochastic blockmodels for directed graphs. J. Amer. Statist. Assoc. 82 8–19.
  • [36] Wu, C. F. J. (1983). On the convergence properties of the EM algorithm. Ann. Statist. 11 95–103.
  • [37] Yao, Y. Y. (2003). Information-theoretic measures for knowledge discovery and data mining. In Entropy Measures, Maximum Entropy Principle and Emerging Applications 115–136. Springer, New York.
  • [38] Zhao, Y., Levina, E. and Zhu, J. (2012). Consistency of community detection in networks under degree-corrected stochastic block models. Ann. Statist. 40 2266–2292.

Supplemental materials