Header

UZH-Logo

Maintenance Infos

Comparing human and algorithm performance on estimating word-based semantic similarity


Batram, Nils; Krause, Markus; Dehaye, Paul-Olivier (2015). Comparing human and algorithm performance on estimating word-based semantic similarity. In: Aiello, Luca Maria; McFarland, Daniel. Social Informatics. Cham: Springer, 452-460.

Abstract

Understanding natural language is an inherently complex task for computer algorithms. Crowdsourcing natural language tasks such as semantic similarity is therefore a promising approach. In this paper, we investigate the performance of crowdworkers and compare them to offline contributors as well as to state of the art algorithms. We will illustrate that algorithms do outperform single human contributors but still cannot compete with results gathered from groups of contributors. Furthermore, we will demonstrate that this effect is persistent across different contributor populations. Finally, we give guidelines for easing the challenge of collecting word based semantic similarity data from human contributors.

Abstract

Understanding natural language is an inherently complex task for computer algorithms. Crowdsourcing natural language tasks such as semantic similarity is therefore a promising approach. In this paper, we investigate the performance of crowdworkers and compare them to offline contributors as well as to state of the art algorithms. We will illustrate that algorithms do outperform single human contributors but still cannot compete with results gathered from groups of contributors. Furthermore, we will demonstrate that this effect is persistent across different contributor populations. Finally, we give guidelines for easing the challenge of collecting word based semantic similarity data from human contributors.

Statistics

Altmetrics

Additional indexing

Item Type:Book Section, refereed, original work
Communities & Collections:07 Faculty of Science > Institute of Mathematics
Dewey Decimal Classification:510 Mathematics
Language:English
Date:2015
Deposited On:14 Jan 2016 10:35
Last Modified:05 Apr 2016 19:41
Publisher:Springer
Series Name:Lecture Notes in Computer Science
Number:8852
ISSN:0302-9743
ISBN:978-3-319-15167-0
Publisher DOI:https://doi.org/10.1007/978-3-319-15168-7_55
Related URLs:http://www.recherche-portal.ch/ZAD:default_scope:ebi01_prod010461545 (Library Catalogue)

Download

Full text not available from this repository.
View at publisher

TrendTerms

TrendTerms displays relevant terms of the abstract of this publication and related documents on a map. The terms and their relations were extracted from ZORA using word statistics. Their timelines are taken from ZORA as well. The bubble size of a term is proportional to the number of documents where the term occurs. Red, orange, yellow and green colors are used for terms that occur in the current document; red indicates high interlinkedness of a term with other terms, orange, yellow and green decreasing interlinkedness. Blue is used for terms that have a relation with the terms in this document, but occur in other documents.
You can navigate and zoom the map. Mouse-hovering a term displays its timeline, clicking it yields the associated documents.

Author Collaborations