Annals of Statistics

Pseudo-likelihood methods for community detection in large sparse networks

Arash A. Amini, Aiyou Chen, Peter J. Bickel, and Elizaveta Levina

Full-text: Open access

Enhanced PDF (1315 KB)

Abstract

Many algorithms have been proposed for fitting network models with communities, but most of them do not scale well to large networks, and often fail on sparse networks. Here we propose a new fast pseudo-likelihood method for fitting the stochastic block model for networks, as well as a variant that allows for an arbitrary degree distribution by conditioning on degrees. We show that the algorithms perform well under a range of settings, including on very sparse networks, and illustrate on the example of a network of political blogs. We also propose spectral clustering with perturbations, a method of independent interest, which works well on sparse networks where regular spectral clustering fails, and use it to provide an initial value for pseudo-likelihood. We prove that pseudo-likelihood provides consistent estimates of the communities under a mild condition on the starting value, for the case of a block model with two communities.

Article information

Source
Ann. Statist., Volume 41, Number 4 (2013), 2097-2122.

Dates
First available in Project Euclid: 23 October 2013

Permanent link to this document
https://projecteuclid.org/euclid.aos/1382547514

Digital Object Identifier
doi:10.1214/13-AOS1138

Mathematical Reviews number (MathSciNet)
MR3127859

Zentralblatt MATH identifier
1277.62166

Subjects
Primary: 62G20: Asymptotic properties
Secondary: 62H99: None of the above, but in this section

Keywords
Community detection network pseudo-likelihood

Citation

Amini, Arash A.; Chen, Aiyou; Bickel, Peter J.; Levina, Elizaveta. Pseudo-likelihood methods for community detection in large sparse networks. Ann. Statist. 41 (2013), no. 4, 2097--2122. doi:10.1214/13-AOS1138. https://projecteuclid.org/euclid.aos/1382547514


Export citation

References

  • [1] Adamic, L. A. and Glance, N. (2005). The political blogosphere and the 2004 US election. In Proceedings of the WWW-2005 Workshop on the Weblogging Ecosystem. ACM, New York.
  • [2] Airoldi, E. M., Blei, D. M., Fienberg, S. E. and Xing, E. P. (2008). Mixed membership stochastic blockmodels. J. Mach. Learn. Res. 9 1981–2014.
  • [3] Amini, A. A., Chen, A., Bickel, P. J. and Levina, E. (2013). Supplement to “Pseudo-likelihood methods for community detection in large sparse networks.” DOI:10.1214/13-AOS1138SUPP.
  • [4] Ball, B., Karrer, B. and Newman, M. E. J. (2011). An efficient and principled method for detecting communities in networks. Phys. Rev. E (3) 34 036103.
    Mathematical Reviews (MathSciNet): MR2788206
    Digital Object Identifier: doi:10.1103/PhysRevE.83.016107
  • [5] Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems. J. R. Stat. Soc. Ser. B Stat. Methodol. 36 192–236.
    Mathematical Reviews (MathSciNet): MR373208
  • [6] Bickel, P. J. and Chen, A. (2009). A nonparametric view of network models and Newman–Girvan and other modularities. Proc. Natl. Acad. Sci. USA 106 21068–21073.
  • [7] Bickel, P. J., Chen, A. and Levina, E. (2011). The method of moments and degree distributions for network models. Ann. Statist. 39 2280–2301.
    Mathematical Reviews (MathSciNet): MR2906868
    Digital Object Identifier: doi:10.1214/11-AOS904
    Project Euclid: euclid.aos/1321020525
  • [8] Bickel, P. J., Choi, D., Chang, X. and Zhang, H. (2012). Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels. Available at arXiv:1207.0865.
    arXiv: 1207.0865
  • [9] Bickel, P. J. and Doksum, K. A. (2007). Mathematical Statistics: Basic Ideas and Selected Topics, 2nd ed. Prentice Hall, New York.
    Mathematical Reviews (MathSciNet): MR443141
  • [10] Celisse, A., Daudin, J.-J. and Pierre, L. (2012). Consistency of maximum-likelihood and variational estimators in the stochastic block model. Electron. J. Stat. 6 1847–1899.
    Mathematical Reviews (MathSciNet): MR2988467
    Digital Object Identifier: doi:10.1214/12-EJS729
    Project Euclid: euclid.ejs/1349355605
  • [11] Channarond, A., Daudin, J. J. and Robin, S. (2011). Classification and estimation in the stochastic block model based on the empirical degrees. Available at arXiv:1110.6517.
    arXiv: 1110.6517
  • [12] Chatterjee, S. (2012). Matrix estimation by universal singular value thresholding. Available at arXiv:1212.1247.
    arXiv: 1212.1247
  • [13] Chaudhuri, K., Chung, F. and Tsiatas, A. (2012). Spectral clustering of graphs with general degrees in the extended planted partition model. JMLR Workshop and Conference Proceedings 23 35.1–35.23.
  • [14] Decelle, A., Krzakala, F., Moore, C. and Zdeborová, L. (2012). Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications. Phys. Rev. E (3) 84 066106.
  • [15] Fortunato, S. (2010). Community detection in graphs. Phys. Rep. 486 75–174.
    Mathematical Reviews (MathSciNet): MR2580414
    Digital Object Identifier: doi:10.1016/j.physrep.2009.11.002
  • [16] Gleser, L. J. (1975). On the distribution of the number of successes in independent trials. Ann. Probab. 3 182–188.
    Mathematical Reviews (MathSciNet): MR365651
    Digital Object Identifier: doi:10.1214/aop/1176996461
  • [17] Handcock, M. S., Raftery, A. E. and Tantrum, J. M. (2007). Model-based clustering for social networks. J. Roy. Statist. Soc. Ser. A 170 301–354.
    Mathematical Reviews (MathSciNet): MR2364300
    Digital Object Identifier: doi:10.1111/j.1467-985X.2007.00471.x
  • [18] Hoeffding, W. (1956). On the distribution of the number of successes in independent trials. Ann. Math. Statist. 27 713–721.
    Mathematical Reviews (MathSciNet): MR80391
    Digital Object Identifier: doi:10.1214/aoms/1177728178
    Project Euclid: euclid.aoms/1177728178
  • [19] Hoff, P. D. (2007). Modeling homophily and stochastic equivalence in symmetric relational data. In Advances in Neural Information Processing Systems, Vol. 19. MIT Press, Cambridge, MA.
  • [20] Holland, P. W., Laskey, K. B. and Leinhardt, S. (1983). Stochastic blockmodels: First steps. Social Networks 5 109–137.
    Mathematical Reviews (MathSciNet): MR718088
    Digital Object Identifier: doi:10.1016/0378-8733(83)90021-7
  • [21] Holland, P. W. and Leinhardt, S. (1981). An exponential family of probability distributions for directed graphs. J. Amer. Statist. Assoc. 76 33–65.
    Mathematical Reviews (MathSciNet): MR608176
    Digital Object Identifier: doi:10.1080/01621459.1981.10477598
  • [22] Karrer, B. and Newman, M. E. J. (2011). Stochastic blockmodels and community structure in networks. Phys. Rev. E (3) 83 016107, 10.
    Mathematical Reviews (MathSciNet): MR2788206
    Digital Object Identifier: doi:10.1103/PhysRevE.83.016107
  • [23] Mariadassou, M., Robin, S. and Vacher, C. (2010). Uncovering latent structure in valued graphs: A variational approach. Ann. Appl. Stat. 4 715–742.
    Mathematical Reviews (MathSciNet): MR2758646
    Digital Object Identifier: doi:10.1214/10-AOAS361
    Project Euclid: euclid.aoas/1280842137
  • [24] Mossel, E., Neeman, J. and Sly, A. (2012). Stochastic block models and reconstruction. Available at arXiv:1202.1499.
    arXiv: 1202.1499
  • [25] Newman, M. E. J. (2004). Detecting community structure in networks. Eur. Phys. J. B 38 321–330.
  • [26] Newman, M. E. J. (2006). Finding community structure in networks using the eigenvectors of matrices. Phys. Rev. E (3) 74 036104, 19.
    Mathematical Reviews (MathSciNet): MR2282139
    Digital Object Identifier: doi:10.1103/PhysRevE.74.036104
  • [27] Newman, M. E. J. (2006). Modularity and community structure in networks. Proc. Natl. Acad. Sci. USA 103 8577–8582.
  • [28] Newman, M. E. J. and Girvan, M. (2004). Finding and evaluating community structure in networks. Phys. Rev. E (3) 69 026113.
  • [29] Newman, M. E. J. and Leicht, E. A. (2007). Mixture models and exploratory analysis in networks. Proc. Natl. Acad. Sci. USA 104 9564–9569.
  • [30] Nowicki, K. and Snijders, T. A. B. (2001). Estimation and prediction for stochastic blockstructures. J. Amer. Statist. Assoc. 96 1077–1087.
    Mathematical Reviews (MathSciNet): MR1947255
    Digital Object Identifier: doi:10.1198/016214501753208735
  • [31] Perry, P. O. and Wolfe, P. J. (2012). Null models for network data. Available at arXiv:1201.5871v1.
    arXiv: 1201.5871v1
  • [32] Rohe, K., Chatterjee, S. and Yu, B. (2011). Spectral clustering and the high-dimensional stochastic blockmodel. Ann. Statist. 39 1878–1915.
    Mathematical Reviews (MathSciNet): MR2893856
    Digital Object Identifier: doi:10.1214/11-AOS887
    Project Euclid: euclid.aos/1314190618
  • [33] Shi, J. and Malik, J. (2000). Normalized cuts and image segmentation. IEEE Trans. Pattern Analysis and Machine Intelligence 22 888–905.
  • [34] Snijders, T. A. B. and Nowicki, K. (1997). Estimation and prediction for stochastic blockmodels for graphs with latent block structure. J. Classification 14 75–100.
    Mathematical Reviews (MathSciNet): MR1449742
    Digital Object Identifier: doi:10.1007/s003579900004
  • [35] Wang, Y. J. and Wong, G. Y. (1987). Stochastic blockmodels for directed graphs. J. Amer. Statist. Assoc. 82 8–19.
    Mathematical Reviews (MathSciNet): MR883333
    Digital Object Identifier: doi:10.1080/01621459.1987.10478385
  • [36] Wu, C. F. J. (1983). On the convergence properties of the EM algorithm. Ann. Statist. 11 95–103.
    Mathematical Reviews (MathSciNet): MR684867
    Digital Object Identifier: doi:10.1214/aos/1176346060
    Project Euclid: euclid.aos/1176346060
  • [37] Yao, Y. Y. (2003). Information-theoretic measures for knowledge discovery and data mining. In Entropy Measures, Maximum Entropy Principle and Emerging Applications 115–136. Springer, New York.
  • [38] Zhao, Y., Levina, E. and Zhu, J. (2012). Consistency of community detection in networks under degree-corrected stochastic block models. Ann. Statist. 40 2266–2292.
    Mathematical Reviews (MathSciNet): MR3059083
    Digital Object Identifier: doi:10.1214/12-AOS1036
    Project Euclid: euclid.aos/1358951382

Supplemental materials

The Institute of Mathematical Statistics

Annals of Statistics

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more