Versatile weighting strategies for a citation-based research evaluation model

Abstract

After a quick review of the most used numerical indicators for evaluating research, we present an integrated model for ranking scientific publications together with authors and journals. Our model relies on certain adjacentcy matrices obtained from the relationship between papers, authors, and journals. These matrices are first normalized to obtain stochastic matrices and then are combined together using appropriate weights to form a suitable irreducible stochastic matrix whose dominant eigenvector provides the desired ranking. Our main contribution is a in-depth analysis of various strategies for choosing the weights, showing their probabilistic interpretation and showing how they affect the outcome of the ranking process. We also prove that, by solving an inverse eigenvector problem, we can determine a weighting strategy in which the relative importance of papers, authors, and journals is chosen by the final user of the ranking algorithm. The impact of the different weighting strategies is analyzed also by means of extensive experiments on large synthetic datasets.