The Annals of Applied Statistics

Empirical assessment of programs to promote collaboration: A network model approach

Katherine R. McLaughlin and Joshua D. EmBree

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


Collaboration networks are thought to be desirable to foster both individual and population productivity. Often programs are implemented to promote collaboration, for example, at academic institutions. However, few tools are available to assess the efficacy of these programs, and very few are data-driven. We carried out a survey at California State University, San Marcos during the 2012–2013 academic year to measure five types of collaboration ties among professors in five science departments at the university over time. During the time period of study, professors participated in NIH-sponsored curriculum development activities with members of other departments. It was hypothesized that participation in these activities would also foster overall collaboration between these departments.

This survey enables the exploration of several methodological and theoretical challenges in network research. In this paper we develop a statistical approach to assess the impact of programmatic interventions on collaboration using model-assisted social network analysis. We derive and implement a hierarchical Bayesian approach to modeling error-prone responses in surveys and examine the effect of an intervention on network structure. Based on this analysis we find an increase in educational collaboration over time after adjusting for the length of time each professor had to form collaborative ties at the university.

Article information

Ann. Appl. Stat. Volume 12, Number 1 (2018), 654-682.

Received: December 2016
Revised: July 2017
First available in Project Euclid: 9 March 2018

Permanent link to this document

Digital Object Identifier

Assessment collaboration dynamic networks error-prone responses grant evaluation hierarchical Bayesian modeling latent network model-assisted analysis recall bias social network analysis survey design


McLaughlin, Katherine R.; EmBree, Joshua D. Empirical assessment of programs to promote collaboration: A network model approach. Ann. Appl. Stat. 12 (2018), no. 1, 654--682. doi:10.1214/17-AOAS1114.

Export citation


  • Borgatti, S. P., Carley, K. M. and Krackhardt, D. (2006). On the robustness of centrality measures under conditions of imperfect data. Soc. Netw. 28 124–136.
  • Butts, C. T. (2003). Network inference, error, and informant (in)accuracy: A Bayesian approach. Soc. Netw. 25 103–140.
  • Butts, C. T., Acton, R. M. and Marcum, C. S. (2012). Interorganizational collaboration in the hurricane Katrina response. J. Soc. Struct. 13.
  • Cross, J. E., Dickmann, E., Newman-Gonchar, R. and Fagan, J. M. (2009). Using mixed-method design and network analysis to measure development of interagency collaboration. Am J. Eval. 30 310–29.
  • Eisenberg, M. and Swanson, N. (1996). Organizational network analysis as a tool for program evaluation. Evaluation & the Health Professions 19 488–506.
  • Frey, B. B., Lohmeier, J. H., Lee, S. W. and Tollefson, N. (2006). Measuring collaboration among grant partners. Am J. Eval. 27 383–392.
  • Gajda, R. (2004). Utilizing collaboration theory to evaluate strategic alliances. Am J. Eval. 25 65–77.
  • Gajda, R. and Koliba, C. (2007). Evaluating the imperative of intraorganizational collaboration: A school improvement perspective. Am J. Eval. 28 26–44.
  • Handcock, M. S. and Gile, K. J. (2010). Modeling social networks from sampled data. Ann. Appl. Stat. 4 5–25.
  • Handcock, M. S., Hunter, D. R., Butts, C. T., Goodreau, S. M., Krivitsky, P. N., Bender-deMoll, S. and Morris, M. (2016). statnet: Software tools for the statistical analysis of network data. The Statnet Project ( R package version 2016.9.
  • Krivitsky, P. N. and Handcock, M. S. (2014). A separable model for dynamic networks. J. R. Stat. Soc. Ser. B. Stat. Methodol. 76 29–46.
  • Loader, C. (2013). locfit: Local regression, likelihood and density estimation. R package version 1.5-9.1.
  • Marcum, C. S., Bevc, C. A. and Butts, C. T. (2012). Mechanisms of control in emergent interorganizational networks. The Policy Studies Journal 40 516–546.
  • Marsden, P. V. (1990). Network data and measurement. Annu. Rev. Sociol. 16 435–463.
  • McCullough, J. (1992). Evaluating grants programs for basic scientific research: Difficult measurement problems, few techniques and slow progress. Am J. Eval. 13 95–101.
  • Snijders, T. A. B. (1996). Stochastic actor-oriented models for network change. J. Math. Sociol. 21 149–172.
  • Wang, D. J., Shi, X., McFarland, D. A. and Leskovec, J. (2012). Measurement error in network data: A re-classification. Soc. Netw. 34 396–409.
  • Woodland, R. H. and Hutton, M. S. (2012). Evaluating organizational collaborations: Suggested entry points and strategies. Am J. Eval. 33 366–383.