Navigation auf zora.uzh.ch

Search ZORA

ZORA (Zurich Open Repository and Archive)

Migration von ZORA auf die Software DSpace

ZORA will change to a new software on 8th September 2025. Please note: deadline for new submissions is 21th July 2025!

Information & dates for training courses can be found here: Information on Software Migration.

Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data

Xu, Shuqi; Mariani, Manuel; Lü, Linyuan; Medo, Matúš (2020). Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data. Journal of Informetrics, 14(1):101005.

Abstract

Despite the increasing use of citation-based metrics for research evaluation purposes, we do not know yet which metrics best deliver on their promise to gauge the significance of a scientific paper or a patent. We assess 17 network-based metrics by their ability to identify milestone papers and patents in three large citation datasets. We find that traditional information-retrieval evaluation metrics are strongly affected by the interplay between the age distribution of the milestone items and age biases of the evaluated metrics. Outcomes of these metrics are therefore not representative of the metrics’ ranking ability. We argue in favor of a modified evaluation procedure that explicitly penalizes biased metrics and allows us to reveal metrics’ performance patterns that are consistent across the datasets. PageRank and LeaderRank turn out to be the best-performing ranking metrics when their age bias is suppressed by a simple transformation of the scores that they produce, whereas other popular metrics, including citation count, HITS and Collective Influence, produce significantly worse ranking results.

Additional indexing

Item Type:Journal Article, refereed, original work
Communities & Collections:03 Faculty of Economics > Department of Business Administration
08 Research Priority Programs > Social Networks
Dewey Decimal Classification:330 Economics
Scopus Subject Areas:Physical Sciences > Computer Science Applications
Social Sciences & Humanities > Library and Information Sciences
Scope:Discipline-based scholarship (basic research)
Language:English
Date:1 February 2020
Deposited On:20 Feb 2020 10:47
Last Modified:21 Jun 2025 02:12
Publisher:Elsevier
ISSN:1751-1577
OA Status:Green
Publisher DOI:https://doi.org/10.1016/j.joi.2019.101005
Other Identification Number:merlin-id:19177
Download PDF  'Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data'.
Preview
  • Content: Accepted Version
  • Language: English
  • Licence: Creative Commons: Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)

Metadata Export

Statistics

Citations

Dimensions.ai Metrics
30 citations in Web of Science®
28 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

128 downloads since deposited on 20 Feb 2020
22 downloads since 12 months
Detailed statistics

Authors, Affiliations, Collaborations

Similar Publications